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FOREWORD 


These  Proceedings  were  prepared  to  present  the  findings 
of  a three-day  Workshop  to  examine  data  from  a compre- 
hensive Air  Force  program  to  determine  the  reliability 
of  in-service  nondestructive  inspection  (NDI ) on  air- 
craft structures.  Workshop  participants  provided  a 
number  of  recommendations  to  enhance  flaw  detection 
reliability  and  courses  of  action  to  implement  these 
recommendations.  A complete  transcription  of  the  eight 
Workshop  Task  Group  reports  which  treated  NDI  problems 
dealing  with  Personnel,  Training,  Operations,  Certifi- 
cation, Equipment,  Reliability  Measurement/Modeling, 
Fracture  Mechanics/NDT  Interrelationships,  and  data 
Analysis  is  included. 

The  Lockheed-Georg ia  Company  was  the  prime  contractor 
for  implementation  of  the  program  to  measure  nonde- 
structive inspection  reliability  in  the  Air  Force  and 
for  making  the  arrangements  for  the  resulting  NDI 
Reliability  Workshop.  The  Lockheed-Georg ia  Company  has 
prepared  these  Workshop  proceedings  and  has  assigned  it 
an  internal  control  number  of  LG73ER0261. 


The  program  sponsors  sincerely  appreciate  the  enthusi- 
astic and  dedicated  efforts  of  all  Workshop  partici- 
pants. We  believe  that  the  technical  community  has 
gained  a valuable  new  insight  into  the  subject  as  a' 
result . 
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Section  I 


INTRODUCTION 


This  report  was  prepared  under  the  authority  of  Con- 
tract F41608-77-D-AC21  to  document  the  proceedings  of 
the  NDI  Reliability  Workshop  sponsored  by  the  U.S.  Air 
Force  Air  Logistics  Command  and  co-directed  by  the  San 
Antonio  Air  Logistics  Center  and  the  Lockheed-Georg i a 
Company  during  the  period  2-4  August  1978.  The  Work- 
shop participants  included  U.S.  Government  and  industry 
personnel  with  representatives  also  attending  from 
Australia,  Canada  and  England.  Initial  results  of  the 
recently  completed  program  titled  "Reliability  of  Non- 
destructive Inspection  on  Aircraft  Structures”  (also 
known  as  "Have  Cracks  - Will  Travel")  formed  the 
technical  data  base  for  the  Workshop.  The  agenda  in- 
cluded comprehensive  presentations  on  the  NDI  Relia- 
bility Program,  reports  on  recently  developed  NDI 
equipment  included  in  the  program  for  evaluation, 
technician  screening  methods,  and  fracture  mechanics 
criteria  applied  to  in-service  inspections,  followed  by 
attendee  participation  in  task  group  studies. 

The  main  body  of  these  Workshop  Proceedings  consists  of 
the  formal  presentations  and  the  findings  and  recom- 
mendations of  the  individual  task  groups  presented  in 
Sections  III  and  V,  respectively. 

The  need  for  an  organized  NDI  program  in  the  Air  Force 
was  conceived  and  implemented  well  over  a decade  ago. 
NDI  has  grown  to  its  present  state  of  development  dur- 
ing the  intervening  period  and  has  contributed  posi- 
tively toward  a reduction  in  maintenance  manhours  and 
aircraft  down  time  as  well  as  improved  operational 
safety  and  aircraft  life  extension.  However,  with  the 
advent  of  damage  tolerance  design  and  maintenance 
philosophy,  there  is  a decided  need  for  achieving 
greater  effectiveness  in  the  application  of  nondest  ac- 
tive testing  and  the  ability  to  measure  effectiveness 
in  the  field. 

Through  the  developing  years  of  nondestructive  testing, 
there  was  no  parallel  development  of  ability  to  express 
results  of  a nondestructive  inspection  in  discrete 
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quantitative  terms.  Past  efforts  were  devoted  to  devel- 
opment of  new  methods,  equipment  and  procedures  to 
enhance  the  capability  to  find  smaller  cracks.  Previ- 
ously, the  question  was;  "what  defects  can  be  found?" 
Now  we  must  ask;  "what  defects  might  remain  unde- 
tected?" This  requirement  led  to  the  initiation  by  the 
Air  Force  Logistics  Command  of  the  "Reliability  of  NDI 
on  Aircraft  Structures"  program  to  determine  the  effec- 
tiveness of  current  NDI  to  detect  various  size  in- 
service  defects  in  aircraft  structure  under  existing 
field  and  depot  conditions. 


The  Reliability  of  NDI  on  Aircraft  Structures  program, 
which  encompassed  a span  of  approximately  four  years, 
included  a planning  stage,  briefings,  preparation  of  a 
detailed  test  plan,  acquisition  and  inspection  of 
cracked  samples,  development  of  NDI  procedures  and  data 
acquisition  forms.  After  a dry  run  at  Wr ight-Patterson 
Air  Force  Base  to  demonstrate  the  entire  program  opera- 
tion, the  field  data  acquisition  phase  was  conducted 
from  October  1975  through  January  1978.  The  data  acqui- 
sition was  accomplished  at  21  Air  Force  Bases  (5  Air 
Logistics  Centers  and  4 bases 
Airlift  Command,  Tactical  Air 
Command,  and  the  Air  Training 
technicians  to  perform  a total 
After  the  data  acquisition  was 
samples  were  subjected  to  a 
provide  baseline  data  (crack)  confirmation.  Preliminary 
program  results  were  combined  into  a data  package  for 
use  by  attendees  at  the  NDI  Reliability  Workshop.  The 
NDI  Reliability  Program  data  results  and  the  presenta- 
tions on  August  2,  1978  describing  the  Reliability 

Program,  constituted  the  technical  basis  for  the  NDI 
Reliability  Workshop. 


each  from  the  Military 
Command,  Strategic  Air 
Command)  using  298  NDI 
of  777  inspection  tasks, 
complete,  the  structure 
teardown  inspection  to 


The  purpose  of  the  Workshop  was  to  present  the  overall 
results  of  the  NDI  Reliability  Program  and  elicit  the 
frank  comments  and  recommendations  from  attendees 
regarding  future  direction  for  improvement  in  NDI 
reliabil ity. 


The  Workshop 
"Reliability 
an  excellent 


was  felt  to  be  a logical  extension  of  the 
of  NDI  on  Aircraft  Structures"  Program  and 
means  of  rapidly  disseminating  the  initial 
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results  of  this  comprehensive  program  which  measured 
the  actual  capability  of  NDI  to  detect  serv ice- induced 
flaws  in  aircraft  structure  under  a maintenance  envi- 
ronment. A workshop  atmosphere  also  permitted  a free 
exchange  of  ideas  and  information  between  industry  and 
the  Air  Force  which  provided  a basis  for  developing  new 
approaches  for  improving  NDI  capability  and  for  a more 
compatible  and  interdependent  relationship  between 
fracture  mechanics  and  NDI  technology. 
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Section  II 
WORKSHOP  FORMAT 


The  NDI  Reliability  workshop  was  organized  to  provide  a 
systematic  medium  for  the  effective  interchange  of  in- 
formation and  experience.  The  complete  Workshop  agenda 
is  included  as  Figure  1.  The  first  day  of  the  workshop 
consisted  of  a series  of  presentations  detailing  the 
complete  program  titled  "Reliability  of  Nondestructive 
Inspection  on  Aircraft  Structures".  The  afternoon 
session  included  presentations  on  the  results  of 
evaluations  of  NDI  equipment  improvements  such  as  the 
automatic  eddy  current  bolt-hole  equipment  and  an 
ultrasonic  scanning  device  for  field  use  which  were 
included  in  the  NDI  Reliability  Program.  A complete 
description  of  the  NDI  Reliability  Program  with  all 
results  is  contained  in  Report  No.  SA-ALC/MME 
76-6-38-1.  A summary  of  presentations  made  during  the 
first  day  of  the  Workshop,  including  slides,  is  given 
in  Section  III. 

The  second  day  was  devoted  to  attendee  participation  in 
Task  Groups  of  their  preference.  Eight  separate  Task 
Groups  had  been  arranged  covering  the  following  areas 
of  interest. 


I 

II 

III 

IV 

V 

VI 

VII 

VIII 


Personnel 

Training 

Operations 

Certification 

Equipment 

Reliability  Measurement/Modeling 
Fracture  Mechani cs/NDI  Interrelationship 
Data  Analysis 


The  intent  of  the  Task  Group  concept  was  to  enable  the 
participant's  background  and  experience,  coupled  with 
the  resources  provided  through  workshop  data  presenta- 
tions and  group  discussions,  to  formulate  decisions  for 
efficient  upgrading  of  NDI.  The  guidelines  provided  the 
participants  were  products  of  both  prior  studies  on  NDE 
Reliability  and  the  Air  Force  Logistics  Command  pro- 
gram, "Reliability  of  NDI  on  Aircraft  Sturctures."  Each 
one  no  doubt  had  additional  subjects  in  mind  which  lent 
themselves  to  the  problem  areas  and  were  encouraged  to 
submit  them  to  their  group  for  consideration  and  inclu- 
sion in  their  recommendations. 
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Government/Industry  Workshop  on  NDI  Reliability 
August  2-4,  1978 
Shamrock  Hilton  Hotel 
Houston,  Texas 


AGENDA 

Wednesday,  August  2,  1978 

8:30  a.m.  to  5:00  p.m. 

Embassy  room 

8:30  a.m. 

Welcome  and  Introduction 
to  Workshop 

B.  W.  Boisvert,  Air  Force 
Logistics  Command 

W.  H.  Lewis 

Lockheed-Georg  ia  Company 

8:45  a.m. 

Background  and  Purpose 
of  Program 

B.  W.  Boisvert 

9:15  a.m. 

Approach  to  Data  Collection 
History 

Samples 

W.  H.  Lewis 

W.  H.  Sproat, 
Lockheed-Georg ia  Company 

Data  Collection 

B.  D.  Dodd, 

Lockheed-Georg  ia  Company 

10:15  a .m. 

Break 

10:45  a .m . 

Teardown  Results 

Overall  Program  Results 

W.  H.  Sproat 

W.  H.  Lewis 

NOON 

Lunch  (Open) 

1:30  p .m . 

Program  Additions 

Automatic  Eddy  Current 
Ultrasonic  "Roto  Scanner" 

Technician  Screening 
Samples 

B.  W.  Boisvert 

B.  W.  Boisvert, 

T.  D.  Cooper,  Air  Force 
Materials  Laboratory 

J.  A.  Moyzis,  Air  Force 
Materials  Laboratory 

2:15  p.m. 

Analysis  of  Program  Results 

W.  H.  Sproat 

3:00  p .m . 

Break 

3:  30  p .m  . 

Fracture  Mechanics  Rqmnts. 

Vs.  Existing  Capability 
AFML  Plans 

T.  D.  Cooper 

J . A . Mo  y z l s 

5:30  - 
7:30  p .m . 

Social  Gathering  - Cash  Bar 

WORKSHOP  AGENDA  (CONT'D) 


Thursday,  August  3.  1978 

8:30  a.m.  Task  Group  Organization 

9:00  a. n.  Fan il iar 1 zat 1 on  with 

Aval  labia  Computer 

9:30  a.m.  Break 


8:30  a.m.  to  9:45  a.m. 
Embassy  room 

T.  D.  Cooper 

K.  D.  Shi  mm  In,  Air  Force 
materials  Laboratory 


10:00  a.m. 

Task  Groups  (Session  I) 

Location 

I. 

Personnel 

Walnut 

II. 

Training 

Belvedere  A 

III. 

Operations 

Boardroom 

IV. 

Cert  1 f icat ion 

Venetian 

V. 

Equipment 

Castilian  C 

VI . 

Reliability  Measurement/ 
Modeling 

Normandy  A 

VII. 

Fracture  Mechanlcs/NDI 
Interrelationship 

Normandy  B 

VIII. 

Data  Analysis 

Castilian  A 

12:30  p.m. 

Workshop 

Luncheon 

Embassy  Room 

Luncheon 

Speaker 

C.  F.  Tiffan: 

Engineering  Con- 
sultant/ Aero- 
nautical Systems 
Division,  Air  Force 
Systems  Command 


1:4  5 p .m. 

3:45  p.m. 
4:00  p ,m . 


Task  Groups  Resume 

(Same  Location,  Session  II) 


Task  Groups  Resume  until  Completion 
(Same  Location,  Session  III) 


Friday,  August  4,  1978 


8:30  a.m.  Task  Group  Reports  and 
Di  scussion 


8:30  a.m.  to  2:00  p.m. 
Castilian  Room 

Task  Group  Leaders 


10:30  a.m.  Break 

11:00  a.m.  Task  Group  Reports  (Cont'd)  Task  Group  Leaders 
1:00  p.m.  Concluding  Summary  B.  W.  Boisvert 


Figure  1.  WORKSHOP  AGENDA  (Sheet  2 of  2) 
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The  central  question  asked  in  each  Task  Group  was:  What 
can  be  done  now  to  improve  NDI  effectiveness? 

Each  Workshop  participant  was  assigned  to  at  least  two 
different  Task  Groups,  based  on  his  preference  and 
professional  interest.  The  opportunity  was  available 
for  the  attendee  to  participate  in  as  many  as  three 
task  groups,  depending  on  his  interests  and  the  con- 
tributions he  wished  to  make  to  the  workshop  objec- 
tives. 

Mr.  Charles  F.  Tiffany,  Engineering  Advisor  for  the 
USAF  Aeronautical  Systems  Division,  Wr ight-Patterson 
AFB , Dayton,  Ohio,  was  the  speaker  for  the  luncheon 
held  during  the  second  day.  Mr.  Tiffany  presented  a 
talk  titled  "The  Role  of  Inspections  in  the  Mainte- 
nance of  Aircraft  Safety"  which  defined  the  needs  and 
challenges  for  NDI. 

All  of  the  data  collected  during  the  conduct  of  the  NDI 
Reliability  Program  had  been  previously  formatted  for 
computerization  and  stored  in  the  ASD  Computer  Center 
at  Wr igh t-Pa t te rson  AFB,  Dayton,  Ohio.  Subsequent 
programs  were  written  for  data  retrieval,  comparison, 
analysis  and  curve  plotting  of  the  large  volume  of 
collected  data.  A remote  computer  terminal  was  in- 
stalled at  the  Workshop  and  connected  via  telephone 
lines  to  the  computer  at  WPAFB  in  order  that  the 
attendees  could  have  rapid  access  to  the  NDI  relia- 
bility data  from  the  program.  This  feature  was  designed 
to  assist  in  discussions  and  forming  recommendations  in 
the  various  Task  Groups. 

The  third  day  of  the  Workshop  was  devoted  to  reports 
from  the  Task  Groups,  recommendations  and  open  discus- 
sion. 

A complete  transcription  of  the  third  day’s  activity  is 
included  in  this  Workshop  Proceeding. 


COMPUTER  STAFF  AT  WORKSHOP  DATA  TERMINAL 


Section  III 

WORKSHOP  PRESENTATIONS 

BACKGROUND  AND  PURPOSE  OF  PROGRAM 

Bernie  W.  Boisvert 
San  Antonio  ALC 

APPROACH  TO  DATA  COLLECTION  - HISTORY 

William  H.  Lewis 
Lockheed-Georg ia  Company 

APPROACH  TO  DATA  COLLECTION  - SAMPLES 

William  H.  Sproat 
Lockheed-Georg ia  Company 

APPROACH  TO  DATA  COLLECTION  - DATA  COLLECTION 

Bruce  Dodd 

Lockheed-Georg ia  Company 


TEARDOWN  RESULTS 

William  H.  Sproat 
Lockheed-Georg ia  Company 

OVERALL  PROGRAM  RESULTS 

William  H.  Lewis 
Lockheed-Georg ia  Company 

PROGRAM  ADDITIONS  - AUTOMATIC  EDDY  CURRENT 

Bernie  W.  Boisvert 
San  Antonio  ALC 

PROGRAM  ADDITIONS  - ULTRASONIC  M ROTO-SCANNER" 

Tom  D.  Cooper 

Air  Force  Materials  Laboratory 

PROGRAM  ADDITIONS  - TECHNICIAN  SCREENING  SAMPLES 

Dr.  J.  A.  Moyzis 

Air  Force  Materials  Laboratory 

ANALYSIS-OF  PROGRAM  RESULTS 

William  H.  Sproat 
Lockheed-Georg ia  Company 


Section  III 


WORKSHOP  PRESENTATIONS  (CONT"D) 

FRACTURE  MECHANICS  REQUIREMENTS  VERSUS  EXISTING  NDI 
CAPABILITY 

Tom  D.  Cooper 

Air  Force  Materials  Laboratory 

FAMILIARIZATION  WITH  AVAILABLE  COMPUTER 

K.  D.  Shimmin 

Air  Force  Materials  Laboratory 


Introduction 


The  remainder  of  this  Section  contains  the  prepared 
text  for  most  of  the  formal  presentations  made  during 
the  first  day  and  second  morning  of  the  Workshop.  These 
are  listed  in  the  preceding  Agenda.  Individual  speakers 
had  the  liberty  of  deviating  from  the  prepared  text; 
hence,  the  text  does  not  necessarily  represent  a ver- 
batim transcript  of  each  talk.  Additionally,  several 
speakers  preferred  to  talk  extemporaneously  from  pre- 
sentation material  supplied  by  themselves  and  a pre- 
pared text  is  therefore,  not  available  for  incorpora- 
tion in  this  Section. 

BACKGROUND  AND  PURPOSE  OF  PROGRAM 
SLIDE  A1 

Good  morning,  gentlemen.  My  name  is  Bernie  Boisvert  and 
in  this  first  talk  of  the  Workshop  agenda  I'm  going  to 
give  you  a brief  overview  of  the  Air  Force  NDI  Relia- 
bility Measurement  Program  that  forms  the  technical 
basis  for  all  of  the  Workshop  activity.  This  program, 
officially  titled  Reliability  Of  Nondestructive  Inspec- 
tion On  Aircraft  Structures  and  unofficially  known  as 
"Have  Cracks  Will  Travel",  is  now  being  completed  by 
the  Contractor,  the  Lo c kheed-G e o r g i a Company  of 
Marietta,  Georgia.  The  program  was  sponsored  primarily 
by  the  Air  Force  Logistics  Command  with  some  guidance 
and  secondary  support  from  the  Air  Force  Materials 
Laboratory.  Mr.  Bill  Lewis  was  the  Lockheed  Program 
Manager;  Mr.  Bill  Sproat  was  the  Program  Principal 
Engineer  and  Mr.  Bruce  Dodd  was  the  Program  Engineer  in 
the  field  who  remained  with  this  program  throughout  its 
data  acquisition  phase.  During  the  day  these  gentlemen 
will  bring  you  some  pertinent  details  concerning  the 
program  and  discuss  the  results  and  analyses  conducted 
to  date.  Other  speakers  from  the  Air  Force  will  discuss 
several  related  topics  and  Air  Force  Materials  Labora- 
tory additions  to  the  program.  From  this  initial  pre- 
sentation this  morning  I will  give  you  a brief  summary 
of  the  program,  discuss  some  of  the  background  that  led 
to  the  program  to  give  you,  I hope,  a clear  understand- 
ing of  the  program  purpose,  the  approach  we  took  to 
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achieve  that  purpose,  and  to  point  out  some  of  the 
major  things  the  results  seem  to  be  telling  us. 

SLIDE  A2 

Nondestructive  inspection  has  been  used  in  the  Air 
Force  now  for  about  a dozen  years.  During  that  time  we 
have  made  significant  progress  in  procuring  and  dis- 
tributing equipment,  training  technicians  and  providing 
detailed  Technical  Order  manuals  on  NDI  applications. 
Because  of  this  progress,  there  have  been  many  examples 
of  reduced  maintenance  manhours  and  reduced  aircraft 
down-time  through  the  application  of  both  base  and 
field  level  NDI.  We  are  all  very  proud  of  our  achieve- 
ment and  believe  that  we  have  the  best  program  pos- 
sible. In  the  past,  however,  and  up  to  now  we  really 
didn't  have  a measure  of  its  effectiveness. 

SLIDE  A3 

Inspection  requirements  can  vary  widely  according  to 
the  type  of  aircraft.  Our  present  NDI  approach  in  the 
Air  Force  is  to  isolate  a specific  problem  area  on  a 
single  component  and  ask  a technician  to  inspect  it. 
Under  these  conditions,  chances  of  finding  a small 
defect,  if  one  exists,  is  usually  very  good.  However, 
we  are  now  finding  that  some  of  our  newer  aircraft 
require  inspections  of  large,  relatively  gross  areas  of 
many  components.  The  probability  that  the  NDI  tech- 
nician will  find  isolated  small  defects  during  the  NDI 
of  numerous  areas  has  until  this  time,  been  virtually 
unknown.  The  problem  is  somewnat  analogous  to  finding  a 
needle  in  a haystack  when  we're  not  sure  if  this  hay- 
stack has  a needle  in  it;  but  if  it  does,  it's  manda- 
tory that  we  find  it. 

SLIDE  A4 

Data  on  the  reliability  and  sensitivity  of  NDI  has  up 
to  now  only  been  developed  for  particular  production 
line  conditions.  Since  the  advent  of  damage  tolerant 
design  philosophy  under  the  MILSTD-1530  family  of 
specifications,  the  Air  Force  has  been  requiring  con- 
tractors of  all  new  programs  to  demonstrate  the  capa- 
bility of  their  production  inspection  processes.  To 
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insure  that  no  flaws  larger  than  those  determined  safe 
by  fracture  mechanics  technology  will  be  present  in  the 
structure  when  it  enters  service,  flaw  detection  curves 
similar  to  these  have  been  generated  by  various  manu- 
facturers under  production  line  conditions  in  their 
plants.  The  conditions  under  which  inspections  are 
normally  conducted  at  both  field  and  depot  facilities 
are  entirely  different  from  the  manufacturing  environ- 
0 ment.  In  the  manufacturing  facility  the  inspector 
usually  looks  at  clean,  bare,  unassembled  parts  and  he 
can  do  this  in  a temperature  and  humidity  controlled 
shop  where  he  can  sit  or  stand  comfortably  at  a bench 
and  turn  or  twist  the  parts  as  necessary  to  get  a com- 
plete inspection. 

SLIDE  5A 

Most  inspections  in  the  field,  on  the  other  hand,  must 
be  accomplished  under  the  whims  of  the  weather  where 
the  part  is  attached  to  or  buried  in  the  structure  and 
the  situation  rarely  permits  the  inspector  to  enjoy 
comfortable  conditions.  Not  only  is  the  method  of  in- 
spection sometimes  drastically  changed,  but  the  types 
of  defects  that  may  exist,  types  of  equipment  and  per- 
sonnel available,  and  accessibility  factors  are  signif- 
icantly altered'.  These  conditions  will  certainly  affect 
the  ability  to  reliably  detect  small  defects.  Not  only 
are  the  conditions  in  the  field  greatly  different  from 
those  in  a production  line  environment  but  they  are 
vastly  different  from  those  existing  in  a laboratory 
environment.  More  often  than  not,  an  engineer  sits  in  a 
laboratory  and  demonstrates  NDI  equipment  response  from 
a flaw  or  machined  slot  that  he  knows  is  there.  The 
technician  in  the  field  is  not  fortunate  enough  to  be 
blessed  with  this  wonderful  knowledge. 

Hence,  it  is  important  to  note  that  this  NDI  Relia- 
bility Measurement  Program  for  the  depot  and  field 
technicians  is  considerably  different  from  previously 
conducted  production  NDI  measurement  and  demonstration 
programs.  The  purpose,  technical  approach  and  results 
of  this  program  certainly  reflect  this  difference.  For 
example,  most  of  the  NDI  evaluation  programs  conducted 
to  date  were  designed  to  demonstrate  the  reliability  of 
selected  production  inspectors  to  detect  specific  size 
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defects  (such  as  a 0.050  inch  crack,  for  example)  . 
These  have  usually  been  conducted  on  a series  of  flat 
specimens,  some  containing  defects,  and  were  inspected 
in  a production  line  manufacturing  environment.  In  our 
program  the  technicians  were  required  to  inspect 
samples  of  actual  built-up  aircraft  structure  contain- 
ing actual  fatigue  cracks  and  were  required  to  inspect 
these  in  the  environment  and  locations  that  they 
normally  perform  their  inspections  in. 

SLIDE  A6 

The  overall  purpose  of  the  AFLC  NDI  Reliability 
Measurement  Program  was  to  determine  the  current 
capabilities  of  NDI  to  detect  various  size  in-service 
defects  in  real  aircraft  structure  under  actual  exist- 
ing field  and  depot  conditions. 

SLIDE  A 7 

Our  approach  to  accomplish  this  purpose  was  to  take 
typical  aircraft  structure  with  known  cracks  to  various 
aircraft  field  and  depot  NDI  shops,  and  have  the  norm- 
ally assigned  personnel  conduct  routine  nondestructive 
inspections.  The  key  point  here  is  that  actual  struc- 
ture with  service  induced  defects  was  taken  to  our 
maintenance  facilities  and  inspected  with  personnel  and 
equipment  normally  used  to  do  these  jobs.  This  approach 
enabled  us  to  accurately  and  r ea 1 i st i ca 1 1 y determine 
the  effectiveness  of  NDI  at  the  operational  level. 
Since  this  had  never  been  done  before,  a great  deal  of 
care  was  taken  at  the  onset  to  carefully  lay  out  and 
plan  a very  thorough  program  that  would  provide  sta- 
tistically valid  data  on  the  many  questions  we  had 
about  our  overall  NDI  program. 

SLIDE  A8 

The  overall  program  was  conducted  in  three  phases  and 
has  taken  us  over  four  years  to  complete.  Phase  I was 
initiated  in  mid  1974  and  consisted  of  a twelve  month 
activity  to  thoroughly  plan  all  of  the  activities  of 
the  program.  This  included  acquiring  the  many  test 
samples  of  actual  hardware  with  known  service  induced 
defects,  to  thoroughly  inspect  these  samples  and 
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catalog  all  of  the  known  defects,  to  develop  a very 
thorough  test  plan  on  how  the  data  from  the  field  were 
to  be  acquired,  develop  the  detailed  inspection  pro- 
cedures, design  all  of  the  forms  on  which  the  data  were 
to  be  recorded,  and  acquire  the  various  transportation 
and  logistics  hardware.  Also  included  in  this  phase  was 
the  preparation  of  a number  of  briefings  regarding  the 
program  and  these  included  not  only  the  detailed  set  of 
instructions  for  the  participants  and  a separate  and 
more  comprehensive  briefing  for  the  local  base  manage- 
ment, but  also  top  level  briefings  for  commanders  of 
each  of  the  major  participating  commands  to  ensure 
their  full  participation  and  support  in  this  program. 
The  plans  for  the  program  were  thoroughly  reviewed  by  a 
panel  of  some  of  the  most  knowledgeable  and  experienced 
individuals  from  both  government  and  industry  in  this 
area  of  technology.  To  ensure  that  nothing  had  been 
overlooked  that  would  invalidate  the  final  results  of 
the  program,  once  all  of  the  planning  and  preparation 
were  complete,  a dry  run  of  the  data  acquisition  was 
made  to  iron  the  wrinkles  out  of  the  program.  This 
trial  dry  run  was  conducted  in  the  summer  of  1975  at 
the  Aeronautical  Systems  Division  4950th  Air  Base  Wing 
at  Wright  Patterson  Air  Force  Base  in  Dayton,  Ohio. 
This  trial  dry  run  served  two  very  important  purposes. 
First  it  enabled  us  to  start  our  data  acquisition  phase 
in  the  field  assured  that  the  program  would  run  smooth- 
ly from  the  start;  and  second,  it  enabled  a large 
number  of  Air  Force  scientists  and  engineers  at  Wright 
Patterson  Air  Force  Base  to  view  the  data  acquisition 
phase  of  the  program  at  first  hand  and  to  make  sugges- 
tions for  potential  improvements.  Next  came  the  series 
of  command  briefings  to  each  of  the  major  participating 
commands  to  assure  their  complete  cooperation  in  the 
program  and  to  schedule  the  visitations  to  their  par- 
ticular bases  and,  finally,  the  actual  data  acquisition 
began  on  14  October  1975.  The  data  acquisiton  phase  of 
the  program  continued  uninterruped  from  the  fall  of 
1975  until  mid  January  1978  when  visitation  of  the  last 
of  the  twenty-one  scheduled  bases  was  completed  at  Shaw 
Air  Force  Base  in  Sumter,  South  Carolina.  At  the  com- 
pletion of  the  data  acquisition  phase,  all  of  the 
samples  were  returned  to  the  contractor's  facility  in 
Marietta,  Georgia  for  a complete  teardown  inspection  to 
verify  the  existence  of  all  suspected  flaws  in  the 
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samples  and  for  an  accurate  destructive  measurement  of 
the  true  size  and  orientation  of  each  crack.  As  you  can 
see  on  the  slide,  we  are  now  in  the  final  phase  of  our 
program,  with  the  Workshop  you  are  currently  attending 
being  one  of  the  vehicles  we  plan  to  utilize  to  help 
disseminate  the  results  of  this  program.  We  had  hoped 
to  be  able  to  provide  you  with  a copy  of  the  final 
report  for  your  use  at  this  Workshop;  however  the  time 
required  to  obtain  all  of  the  necessary  approvals  of 
such  a comprehensive  report  has  precluded  its  final 
release  as  of  this  date.  Instead,  you  have  been  pro- 
vided with  a working  copy  of  the  majority  of  the 
material  that  will  be  contained  in  the  final  report. 
And  also  as  you  can  see,  not  only  will  all  of  the 
attendees  at  this  Workshop  receive  a copy  of  the  final 
report  once  it  is  approved  and  published,  but  you  will 
also  be  receiving  a copy  of  the  Workshop  Proceedings 
later  this  year. 


SLIDE  A9 

Now  that  you  have  a brief  idea  of  what  the  NDI  Relia- 
bility Program  was  all  about  and  how  it  was  conducted, 
let  me  point  out  some  implications  of  the  results,  then 
I would  like  to  define  a few  areas  where  the  total 
overall  NDI  Program  needs  your  help  and  what  we  hope  to 
accomplish  at  this  Workshop.  First  of  all,  the  results 
tell  us  that  the  average  NDI  capability  in  the  Air 
Force  is  not  as  good  as  we  might  desire  it  to  be.  We 
must  now  try  to  identify  the  true  reasons  for  this  and 
to  identify  the  positive  steps  we  can  take  to  improve 
it.  Another  thing  we  must  do  is  to  get  structural 
designers,  structural  analysts  and  inspection  personnel 
all  talking  to  each  other  in  a compatible  language 
based  on  realistic  inspection  requirements  and  capa- 
bilities. The  question  that  you  see  on  this  slide  is  a 
very  pertinent  one  that  we  want  you  to  take  very 
seriously.  We  solicit  your  ideas  and  suggestions  toward 
improving  nondestructive  inspection  in  every  area.  We 
need  your  help  in  looking  for  ways  to  improve  our  NDI 
capability  without  resorting  to  costlier  or  lengthier 
inspection  approaches.  Without  adopting  a market  basket 
of  approaches,  some  of  which  may  not  really  buy  us 
anything.  What  approaches  to  improvement  in  NDI  promise 
the  greatest  cost  effectiveness?  What  causes  and 
effects  do  you  see  in  our  data?  What  implications  can 


you  see  there?  From  your  viewpoint  of  experience  what 
approaches  bring  us  the  greatest  benefits?  Do  these 
approaches  involve  personnel  qualification  and  certi- 
fication, revamping  our  training  programs,  better 
initial  selection  of  and  retention  of  our  NDI  person- 
nel, developing  new  and  improved  equipment,  greater 
sensitivities,  different  inspection  planning? 

Because  of  the  unique  contributions  that  can  be  made 
not  only  to  operational  NDI  but  also  to  NDI  training, 
equipment,  and  to  the  design  and  serviceability  of  all 
types  of  hardware,  we  cannot  overemphasize  the  poten- 
tial importance  of  this  program  nor  the  importance  of 
contributions  you  will  have  the  opportunity  to  make 
during  this  meeting.  We  have  right  here  in  this  room 
perhaps  the  greatest  collection  of  experience  and 
expertise  in  nondestructive  inspection.  We  want  you  to 
bring  that  experience  and  expertise  to  bear  on  the 
questions  we  have  raised.  We  want  you  to  share  your 
knowledge  and  experience  so  that  collectively,  we  may 
provide  the  guidance  and  direction  toward  overall 
improvement  in  our  NDI  capability. 

One  final  comment  from  me.  We  also  want  you  to  take  a 
look  at  our  NDI  reliability  program.  What  would  you 
have  done  differently  if  you  were  going  to  undertake 
such  a capability  assessment  program?  What  are  the 
inherent  limitations  for  applying  the  results  of  the 
program?  Has  there  been  anything  omitted  from  this 
program  that  should  be  considered  for  inclusion  in 
future  programs?  In  other  words,  just  how  good  or  how 
useful  is  our  program?  We  would  like  to  have  your 
opinion.  Now  that  I hope  I have  given  you  a brief 
insight  into  the  background  and  overall  purpose  of  our 
program,  I am  going  to  let  the  contractor  fill  you  in 
with  the  details  of  exactly  how  the  program  v/as  con- 
ducted. First  Bill  Lewis  will  give  you  some  of  the 
history  behind  the  program  and  an  insight  into  the 
planning  efforts  that  went  into  the  program.  Next  Bill 
Sproat  will  describe  the  details  of  each  of  the  samples 
that  we  used  in  the  program  and  finally  Bruce  Dodd,  the 
project  engineer  that  accompanied  the  program  through- 
out the  entire  data  acquisition  effort  of  over  two 
years,  will  give  you  the  details  on  exactly  how  the 
data  were  collected  at  all  of  the  Air  Force  installa- 
tions he  visited.  Thank  you. 


Purpose 


APPROACH  TO  DATA  COLLECTION  - HISTORY 
SLIDE  B1 


Good  morning.  I want  to  offer  you  a brief  account  of 
some  of  the  historical  factors  that  shaped  this  NDI 
reliability  measurement  program  to  its  final  form.  This 
program  is  only  a part  of  the  large  body  of  thought  and 
experimentation  that  began  to  take  shape  in  the  late 
1960's  and  now  after  more  than  a decade  is  beginning  to 
point  us  to  some  of  the  necessary  remedies  for  achiev- 
ing a much  greater  effectiveness  in  the  application  of 
nondestructive  testing. 

SLIDE  B2 

The  NDT  community  has  long  been  aware  of  the  inability 
to  express  the  results  of  a nondestructive  examination 
in  discrete  quantitative  terms.  Hence,  there  have  been 
continuous  efforts  to  develop  new  methods,  equipment 
and  procedures  that  will  provide  a more  quantitative 
measure  of  the  defects  in  the  material.  One  major 
factor  that  has  been  missing  is  the  knowledge  of  how 
effective  our  methods  and  procedures  really  are  in  any 
particular  application.  Previously,  NDI  effectiveness 
was  accepted  mostly  on  the  basis  of  what  defects  were 
actually  found.  In  most  cases  it  was  assumed  and  some- 
times disastrously  that  whenever  NDT  was  applied  to  a 
material  or  a part  the  results  gave  an  accurate  assess- 
ment of  the  part's  integrity.  Obviously  this  was  a 
mistake.  What  was  not,  of  course,  taken  into  account, 
probably  because  it  was  invisible  or  unknowable  at  the 
time,  is  what  defects  remained  undetected  in  the  part 
at  the  conclusion  of  the  inspection.  This,  I submit,  is 
the  most  crucial  factor  in  an  NDT  examination. 

SLIDE  B3 

To  put  this  another  way,  our  efforts  have  been  seem- 
ingly directed  towards  what  is  the  smallest  crack  we 
can  find.  How  many  times  have  you  heard  the  same 
question  yourself?  Perhaps  you  have  asked  this  question 
many  times.  How  small  of  a crack  can  you  find?  But  of 
course  this  isn't  really  what  we  want  to  know.  We  don't 
really  care  what  the  smallest  crack  is  we  can  find. 
Some  of  our  present  day  equipment,  under  the  right 
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circumstances,  has  the  capability  of  finding  extremely 
small  defects. 

SLIDE  B4 


The  question  we  really  ought  to  be  asking,  of  course, 
is  "what  is  the  largest  crack  that  can  be  missed?"  We 
don't  ever  ask  this  question.  Perhaps  that  is  because 
we  realize  we  don't  really  know  the  answer.  But  this  is 
actually  what  we  need  to  know.  After  we  have  applied 
the  latest  that  our  technology  has  to  offer  in  the 
inspection  and  examination  of  a part  or  material,  what 
is  the  largest  defect  that  remains  undetected?  A 
detected  defect  can  either  be  repaired  or  eliminated. 
Tt  Ts  tTie  undetected  defect  which  remains  in  spite  of 
all  of  our  good  intentions  that  can  kill  us  if  it  gets 
big  enough. 

SLIDE  B5 

This  is  perhaps  the  most  famous  crack  in  modern  his- 
tory. An  F-lll  aircraft  was  lost  while  on  a training 
mission  from  Nellis  AFB , Nevada,  in  1969.  Although  the 
aircraft  only  had  105  total  flight  hours,  it  experi- 
enced catastrophic  failure  of  the  wing  box.  The  cause 
of  the  accident  was  determined  to  be  a nearly  one  inch 
long  flaw  on  the  surface  of  the  lower  panel  in  the  D6AC 
wing  carry-through  box.  This  flaw,  which  had  been  over- 
looked during  production  inspection,  was  about  90% 
through  the  entire  part  thickness.  The  light  band  shown 
here  surrounding  the  flaw  is  the  region  of  fatigue 
propagation  which  took  place  during  the  105  flight  hour 
life  of  the  aircraft  prior  to  the  accident.  The  point 
that  I want  to  illustrate  is,  that  this  inch  long-flaw 
escaped  detection  during  manufacturing  inspection  which 
is  certainly  under  more  ideal  environmental  and  acces- 
sibility conditions  than  most  maintenance  inspection 
operations.  Therefore,  it  can  only  be  assumed  that 
possibly  similar  type  defects  could  be  overlooked  and 
missed  during  a typical  maintenance  inspection. 
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SLIDE  B6 

I would  like  to  quickly  point  out  that  I am  not  singl- 
ing out  the  F-lll  incident;  but  it  certainly  serves  to 
illustrate  the  point  that  rather  large  and  seemingly 
obvious  defects  can  be  missed  and  remain  undetected  at 
the  completion  of  a nondestructive  inspection.  As  shown 
here,  tool  damage  in  a fastener  hole  that  went  unde- 
tected, caused  this  wing  failure  on  an  F-5  aircraft  in 
April  of  1970. 

SLIDE  B7 

In  197  3,  a wing  attachment  lug  on  an  F-4  aircraft 
failed  due  to  an  undetected  fatigue  crack  that  was 
missed  during  routine  nondestructive  inspection,  and 
within  six  flights  the  adjacent  wing  attachment  lug 
failed,  resulting  in  the  loss  of  the  entire  wing  and, 
of  course,  the  aircraft.  I could  continue  to  go  on  and 
on  citing  example  after  example  using  the  C-5,  C-130, 
KC-135,  B-52,  and  many  other  aircraft  where  flaws  and 
cracks  have  been  missed  during  routine  inspections  and 
the  results  have  been  catastrophic. 

SLIDE  B8 

I certainly  don't  want  to  leave  you  with  the  impression 
that  aircraft  inspectors  are  the  only  ones  who  miss 
cracks.  Shown  here  is  the  disastruous  derailment  of 
ftntrak  Train  No.  40. 

SLIDE  B9 

This  detailment  was  attributed  to  the  rail  failure 
shown  here,  which  was  caused  by  a 14  inch  long  crack 
which  had  been  missed  in  a recent  rail  car  inspection 
of  this  particular  section  of  track.  I think  it  is 
sufficient  to  say  that  we  have  ample  reason  to  question 
the  reliability  of  our  present  day  nondestructive 
inspections  to  find  all  of  the  cracks  that  we  have  to, 
in  order  to  prevent  structural  failures. 
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By  using  fracture  mechanics  technology,  we  can  predict 
what  size  defects  our  structures  can  safely  tolerate 
and  how  fast  those  defects  will  grow  to  unsafe  sizes 
under  operational  conditions  in  service.  What  we  don't 
know  is  how  large  those  defects  have  to  be  before  we 
can  be  sure  that  we  will  find  them  during  a properly 
conducted  inspection.  This  was  the  overriding  drive 
behind  the  program  we  have  just  completed  for  the  Air 
Force,  ’’nder  typical  field  and  depot  maintenance  con- 
ditions, what  size  cracks  in  aircraft  structure  can  be 
reliably  found  and  what  size  cracks  could  be  missed? 

SLIDE  Bll 

It  was  determined  that  the  most  direct  and  cost  effec- 
tive approach  to  resolve  this  question  was  to  take 
typical  aircraft  structure  with  existing  service- 
induced  cracks  to  various  Air  Force  bases  and  depots 
and  have  the  normally  assigned  NDI  Technicians  go  about 
conducting  nondestructive  inspections  on  these  samples, 
using  their  existing  equipment  and  procedures  typical 
of  everyday  useage. 

SLIDE  B12 

Our  initial  program  contained  two  different  types  of 
aircraft  structure,  and  we  tentatively  planned  to 
conduct  sample  inspections  at  all  five  Air  Force 
Logistic  Command  depots  and  six  typical  bases  in  the 
field.  As  Bernie  mentioned,  a critical  review  of  the 
original  plan  for  conducting  the  program  was  conducted 
in  November  of  1974  by  a select  Steering  Committee 
composed  of  various  government  and  industry  experts. 
This  sixteen  member  committee  conducted  a full  review 
of  past  programs  of  this  type  and  made  some  important 
decisions  as  to  the  approach  this  program  should  take 
in  order  to  assure  that  the  results  would  provide  a 
full  and  valid  evaluation  of  Air  Force  NDI  capabilities 
and  that  the  results  would  be  not  only  statistically 
valid,  but  generally  accepted  by  all  concerned.  This  is 
certainly  not  the  type  of  program  that  can  be  conducted 
every  day,  not  even  every  year  or  so,  and  it  was  there- 
fore extremely  important  to  be  sure  that  everything  was 
right  from  the  start. 
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SLIDE  B13 


The  steering  committee  recommended  an  expansion  of  the 
types  and  quantity  of  program  samples  to  cover  a 
greater  range  of  aircraft  structure  and  an  increase  in 
the  number  of  flaws  in  each  size  range.  The  number  of 
types  of  samples  in  the  program  increased  from  two  to 
tfie  six  shown  here  which  typify  various  components  of 
an  aircraft  structure  which  require  periodic  nonde- 
structive inspection.  I would  like  to  point  out  that  it 
was  no  small  effort  to  obtain  suitable  test  samples.  It 
was,  of  course,  necessary  to  provide  a range  of 
specimens  that  were  fairly  representative  of  aircraft 
structure  inspected  in  various  commands.  All  samples 
would  have  to  contain  actual  fatigue  cracks  in  a wide 
range  of  sizes.  We  were  fortunate,  in  that  a number  of 
wing  boxes  remained  from  a C-130  inner  wing  box  re- 
placement program  a few  years  earlier  which  provided 
many  of  the  final  structural  samples.  These  wing  boxes, 
whicii  had  exceeded  their  originally  designed  fatigue 
life  contained  a number  of  typical  fatigue  cracks.  In 
addition,  one  of  the  boxes  removed  from  a commercial 
C-130  had  undergone  considerable  full-scale  fatigue 
testing  which,  in  fact,  had  generated  a number  of  addi- 
tional cracks.  A number  of  titanium  truss  specimens 
containing  a variety  of  fatigue  cracks  were  available 
from  a previously  conducted  C-5A  engine  pylon  NDI 
reliability  assessment  program  and  we  were  able  to 
utilize  those  in  this  program.  With  the  exception  of  a 
few  other  components  derived  from  some  selected  full- 
scale  fatigue  testing,  actual  aircraft  structure 
containing  service  induced  fatigue  cracks  was  very 
difficult  to  come  by.  The  Steering  committee,  through 
their  own  contacts,  made  a maximum  effort  to  obtain 
what  ever  scrapped  aircraft  components  were  readily 
available  during  this  time  period  that  contained  real 
life  in-service  fatigue  cracks. 

As  a result,  the  small  number  of  samples  of  KC-135 
center  wing  structure  and  a number  of  forged  fittings 
from  a F-104  wing  were  obtained  and  included  in  the 
program.  I think  it  is  important  here  to  note  something 
that  you  may  find  interesting  and  which  I think  serves 
to  further  emphasize  several  of  the  points  we  have  been 
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making.  The  KC-135  wing  panel  specimens  that  we  in- 
cluded in  our  program  were  originally  found  cracked  by 
Boeing  during  the  KC-135  replanking  program  at  their 
Wichita,  Kansas  facility.  The  cracked  panels  were 
forwarded  by  Boeing  to  the  Air  Force  Logistics  Center 
at  Oklahoma  City  where  the  cracks  were  verified  and  the 
panels  stored.  We  obtained  the  panels  from  Oklahoma 
City,  reinspected  them,  and  not  only  noted  the  exact 
location  of  each  crack  but  also  made  an  estimate  of  its 
apparent  size  and  orientation.  Structural  sample  speci- 
mens were  then  made  from  these  panels  and  included  in 
our  program  which  toured  the  twenty-one  Air  Force  in- 
stallations visited.  During  that  time,  several  hundred 
Air  Force  technicians  inspected  these  panels  and  a 
number  of  them  found  the  small  cracks  that  were  sup- 
posedly contained  in  these  samples.  I used  the  word 
supposedly  because,  as  you  will  later  see,  in  order  to 
verify  the  true  size  and  shape  of  each  and  every  crack 
in  our  program,  all  of  the  structural  samples  were 
destructively  disassembled  and  metallurg ically  examined 
under  careful  laboratory  conditions.  Not  any  of  the 
KC-135  wing  panels  we  included  in  our  program  contained 
a single  crack!  Again,  these  "cracks"  had  been  found  by 
Boeing,  verified  by  the  Air  Force,  and  not  only  re- 
verified by  Lockheed,  but  we  even  measured  their 
apparent  size.  Now  admittedly  we  thought  these  were 
pretty  small  to  begin  with,  most  of  them  measuring  less 
than  .050  in  length;  nevertheless,  we  had  detected  what 
we  thought  were  apparent  fatigue  cracks.  In  rea  ity 
most  of  them  turned  out  to  be  either  scratches,  gouges, 
or  pits  in  and  around  the  fastener  holes  that  were 
produced  by  disassembly  and  subsequent  cleaning  and 
etching  operations  prior  to  their  initial  inspection. 
But  I think  this  does  serve  to  point  out  part  of  our 
reliability  problem  with  nondestructive  inspection. 

SLIDE  B14 

Shown  here  are  the  four  basic  methods  of  nondestructive 
inspection  that  we  evaluated  in  our  program.  The  one 
method  that  is  conspicious  by  its  absence  in  the  pro- 
gram is  the  use  of  magnetic  particle.  There  were  sev- 
eral reasons  for  this  and  after  thorough  evaluation  the 
steering  committee  determined  that  it  was  next  to  im- 
possible, during  the  time  frame  that  we  had,  to  obtain 
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a large  enough  quantity  of  steel  samples  with  a range 
of  crack  sizes  necessary  to  include  in  our  program. 
Since  most  of  the  high  strength  steels  used  in  the 
aircraft  industry  do  not  tolerate  large  cracks,  most 
flawed  steel  parts  fail  before  the  flaw  has  grown  to  a 
sufficient  size  to  enable  easy  detection. 

SLIDE  B15 

Finally,  because  of  the  large  amount  of  variation 
between  the  different  commands  and  their  differences  in 
applying  NDI , not  only  between  the  commands  but  between 
various  bases  within  each  command,  the  Steering  Com- 
mittee determined  that  a broader  sampling  of  field 
installations  was  necessary.  Consequently  the  scope  of 
the  program  was  enlarged  to  include  not  only  all  five 
Air  Force  depots  but  a minimum  of  four  bases  within 
each  of  the  four  major  operating  commands  of  TAC,  MAC, 
SAC,  and  the  Air  Training  Command. 

SLIDE  B16 

Once  the  steering  committee  had  approved  the  overall 
program  approach,  the  detail  preparation  for  acquiring 
the  data  began  in  earnest.  This  included  not  only  as- 
sembling and  preparing  all  of  the  stuructural  samples 
and  cataloging  their  crack  locations  , and  apparent 
sizes,  but  developing  detailed  straight-forward  NDI 
procedures  identical  to  those  that  the  technicians  were 
accustomed  to  using  in  his  Tech  Order,  for  his  use  in 
inspecting  and  hopefully  detecting  the  cracks  in  the 
samples.  We  also  designed  a series  of  data  collection 
forms  that  would  enable  us  to  not  only  record  whether 
or  not  the  inspecor  found  the  cracks  in  the  samples  and 
which  ones  he  missed,  but  forms  were  also  designed  to 
record  as  many  of  the  variables  as  we  could  that  were 
associated  with  our  data  collection. 

These  will  be  discussed  in  some  detail  with  you  later 
this  morning  so  that  you  will  have  an  insight  into  the 
vast  amount  of  data  that  we  actually  have  collected  and 
is  currently  stored  in  the  computer  files  at  the  Aero- 
nautical Systems  Division  Computer  Center  at  Wright- 
Patterson  Air  Force  Base. 
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SLIDE  B17 

All  in  all,  the  planning  phase  of  our  program  took  just 
about  a year  to  complete  and  resulted  in  a very  com- 
plete and  comprehensive  Program  Test  Plan.  The  Phase  I 
Test  Plan  Report,  of  which  we  have  several  copies  here, 
is  available  through  the  government  information  ex- 
change and  the  Defense  Documentation  Center  data  sys- 
tems. It  contains  all  of  the  details  about  exactly  how 
the  program  was  conducted,  including  samples  of  the 
data  forms  that  we  used  to  collect  the  data,  detailed 
descriptions  of  each  of  the  specimens  used,  the  sched- 
ule for  all  of  the  installations  visited,  a copy  of  the 
NDI  manual  containing  the  detailed  inspection  pro- 
cedures developed  for  the  program  that  the  technicains 
used  to  inspect  the  specimens  and  a copy  of  the  orien- 
tation briefing  that  was  given  to  each  technician  that 
participated  in  the  program.  Finally*  after  all  of  the 
planning  was  completd,  as  Bernie  mentioned,  we  con- 
ducted briefings  at  the  headquarters  of  each  of  the 
major  participating  commands,  not  only  to  brief  them  on 
exactly  what  the  program  was  about  and  why  it  was  being 
conducted,  but  to  insure  that  we  had  the  full  coopera- 
tion of  each  of  the  participating  commands  to  permit  us 
on  their  base  and  supply  the  manhours  for  the  techni- 
cians to  participate  in  our  program.  Now  before  we 
discuss  the  data  collection  phase  of  the  program,  I 
think  it  is  important  that  each  of  you  get  fixed  in 
your  mind  exactly  what  each  of  the  specimens  we  have  in 
the  program  were  and  exactly  what  the  NDI  inspections 
consisted  of  and  the  detailed  inspection  procedures 
that  were  used.  Bill  Sproat  will  now  give  you  a brief 
rundown  on  each  of  these. 
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SLIDE  B5 

ROGUE  INITIAL  CRACK  IN  F-111 


SLIDE  B15 
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Data  Acquisition  Visitations 

• i AlC  DEPOTS 

• 4 TAC  BASES 

• 4 MAC  BASES 

• 4 SAC  BASES 

• 4 ATC  BASES 


SLIDE  B17 


APPROACH  TO  DATA  COLLECTION  - SAMPLES 


SLIDE  Cl 

The  actual  aircraft  structures  with  cataloged  fatigue 
cracks  will  be  shown  in  the  following  slides  with  the 
NDI  in-progress  on  each.  All  inspections  were  aimed  at 
detection  of  flaws  radiating  from  fastener  sites,  plus 
several  drain  holes  in  the  first  two  sample  types  in 
the  series.  There  was  approximately  a ten-to-one  ratio 
of  flaws-to- inspection  sites  in  these  samples.  Each 
crack  was  treated  individually  where  damage  progressed 
from  both  sides  of  a fastner  hole;  so  a cataloged  flaw 
record,  for  example,  would  have  flaw  101A  and  101B 
listed  for  each  side  of  the  fastener  hole.  The  NDI  task 
size  in  most  cases  was  designed  to  occupy  approximately 
8 hours,  but  no  time  limits  were  placed  on  the 
individual. 

SLIDE  C2 

SAMPLE  AET,  GELAC 

This  intact,  painted  segment  of  a C-130  center  wing  box 
was  designated  sample  A.  A 2-foot  wide  by  6-foot  long 
chordwise  strip  containing  528  fasteners  and  14  drain 
holes  comprised  the  inspection  sample.  Here,  the  trials 
for  the  eddy  current  procedure  validation  is  being 
made . 

SLIDE  C3 
SAMPLE  AET,  AF 

This  procedure  involves  circumferential  scans  around 
the  sites  with  the  aid  of  a circle  template.  There  were 
originally  40  individual  flaws  cataloged  in  this  sample 
by  a very  close  inspection  at  Lockheed-Georg  i a using 
the  circle  template. 


SLIDE  C4 


SAMPLE  AUT,  GELAC 

The  same  Sample  A was  inspected  by  ultrasonic  shear 
wave  scans  with  the  optical  aid  of  a straight-edge  to 
perform  spanwise  scans  to  detect  chordwise  cracks. 

SLIDE  C5 

SAMPLE  AUT,  AF 

Most  participants  didn't  elect  to  use  the  straightedge 
guide  because  it  tended  to  inhibit  essential  scanning 
motions . 

SLIDE  C6 


SAMPLE  BET,  GELAC 

The  composition  of  Sample  B was  virtually  identical 
with  Sample  A except  that  it  was  made-up  from  12  pieces 
of  painted  wing  cover  with  stringers  attached.  These 
segments  were  approximately  18"  square,  with  four  of 
them  removed  from  the  perimeter  of  a wing  access  cutout 
area.  Eddy  current  surface  scans  which  were  identical 
to  those  on  Sample  A were  being  validated  here. 

SLIDE  C7 

SAMPLE  BET (OH),  GELAC 

This  sample,  composed  of  the  12  pieces  in-total,  con- 
tained 742  fastener  sites  with  51  cataloged  flaws. 
Here,  the  overhead  NDI  position  is  being  checked-out  in 
a dry  run.  Both  bench-top  and  overhead  procedures  were 
employed  in  the  data  acquisition  phase  to  determine  the 
effect  of  position  on  results. 

SLIDE  C8 

SAMPLE  BET (OH),  AF 

In  some  cases,  a team  conducted  the  overhead  eddy 
current  surface  scans  to  facilitate  ease  of  inspection. 
We  will  be  showing  the  comparison  of  results  of  bench 
top  versus  overhead  in  the  overall  results,  later. 


SLIDE  C9 


SAMPLE  BRT , GELAC 

Radiographic  NDI  was  performed  on  the  B sample  with  the 
equipment  trailer  serving  as  a simulated  fuselage. 
Exposures  were  made  individually  with  the  segments 
positioned  on  a series  of  racks  inside  the  trailer. 
Double  film  cassette  loads  were  made  for  most  exposures 
to  accommodate  different  thicknesses  of  material. 

SLIDE  CIO 

SAMPLE  BRT,  AF 

Almost  all  radiography  exposures  were  conducted  by 
pairs  of  individuals,  as  is  commonly  done  in-practice. 
Film  reading  was,  however,  performed  by  only  one  of  the 
pair  in  most  cases. 

SLIDE  Cll 
SAMPLE  CUT,  GELAC 

Sample  C was  composed  of  26  bare  titanium  segments 
nestted  between  stringers  inside  the  intact  box,  Sample 
A.  Ultrasonic  shear  wave  scans  were  performed  near  both 
upper  and  lower  edges  on  both  faces  of  each.  A total  of 
354  sites,  configured  as  half-holes,  were  examined  with 
54  flaws  in  that  total. 

SLIDE  C12 
SAMPLE  CUT,  AF 

The  ultrasonic  procedure  provided  tor  an  optional 
transducer  guide  to  assist  in  positioning.  This  guide, 
however  , was  not  used  in  most  cases  because  it  in- 
hibited scanning  motion  which  the  individual  sought  to 
use  in  the  inspection. 


A 


SLIDE  C13 
SAMPLE  CPT,  AF 

The  penetrant  NDI  on  the  C Sample  was  conducted 
alternately  with  ultrasonic  NDI  with  oily  couplant,  to 
present  a cleaning  problem  similar  to  in-service  con- 
ditions. Conditions  within  the  wing  box  simulated  wing 
riser  inspection. 

SLIDE  C14 
SAMPLE  DUT  GELAC 

Sample  D was  composed  of  two  painted  KC-135  wing  panel 
segments  with  fasteners  removed.  A total  of  105  holes, 
of  which  10  were  alledgedly  cracked,  was  inspected  by 
ultrasonic  shear  wave  scanning  around  the  perimeter. 

SLIDE  C15 

SAMPLE  DET,  GELAC 

These  same  panel  segments  were  inspected  by  eddy 
current  bolt-hole  scans. 

SLIDE  C16 

SAMPLE  DET,  AF 

The  field  task  was  commonly  performed  as  a bench-top 
activity. 

SLIDE  C17 
SAMPLE  EET , GELAC 

Sample  E was  composed  of  a number  of  F-104  forged  wing 
fittings  attached  to  dummy  cover  plates.  All  fastener 
holes  were  open  and  eddy  current  bolt-hole  NDI  was 
performed  on  a total  of  140  holes.  Of  these,  27  were 
cataloged  as  being  potentially  flawed. 


SLIDE  C18 


SAMPLE  EET,  AF 

This  NDI  task  was  also  commonly  peformed  as  a bench-top 
inspection  in  the  field. 

SLIDE  C19 

SAMPLE  FET,  GELAC 


This  structure.  Sample 
figuration  containing  a 
and  stiffener  elements, 
to  evaluate  a fastener 
holes  composed  the  inspection  sites.  Here,  the  eddy 
current  bolt-hold  scan  procedure  is  being  validated  for 
detection  of  53  potential  flaws. 


F,  was  a C-5  wing  spar  con- 
symmetric  assembly  of  cap,  web 
It  was  used  in  a fatigue  test 
design.  A total  of  257  open 


SLIDE  C20 


SAMPLE  FET,  AF 


The  inspection  areas  on 
positioned  on  each  side 
all  performed  from 


the 


this  structure  were  vertically 
The  bolt-hole  inspections  were 
ex  ter i or  . 


SLIDE  C21 


SAMPLE  FUT,  GELAC 


Ultrasonic  NDI  on 
both  the  interior 
using  a shear  wave 


this  structure  required  access  to 
and  exterior,  with  scans  performed 
technique  , 


SLIDE  C22 
SAMPLE  FUT,  AF 


Ultrasonic  shear  wave  scans  on  the  exterior  were  made 
to  detect  cracks  in  the  0.060"  thick  web,  under  the 
vertical  stiffeners,  while  those  on  the  interior  were 


conducted  on  the  0. 
ture  presented  the 
among  all  structure 


25"  thick  extruded  cap.  This  struc- 
most  difficult  challenge,  for  NDI 
samples  used  in  the  program. 


SLIDE  C23 


SAMPLE  GET,  AF 


A new  sample  type  was  introduced  at  the  18th  base,  for 
use  throughout  the  remainder  of  the  data  acquisition 
effort.  A total  of  76  specimens  1/4"  thick,  2"  wide  by 
16"  long,  containing  ten  countersunk  fastener  holes, 
3/16"  diameter,  were  designated  as  Sample  G.  These 
specimens  contained  a total  of  1520  machine  grown 
fatigue  cracks  at  randomly  located  holes.  Inspections 
were  conducted  by  ultrasonic  shear  wave  and  eddy- 
current  bolt  hole  scans  to  detect  the  flaws. 


The  purpose  of  this  was  to  determine  for  the  Air  Force 
Materials  Laboratory  what  relationship  exists  between 
NDI  performance  on  simple  flat  plates  with  machine 
grown  flaws  and  assembled  structure  with  service 
induced  flaws.  Joe  Moyzis  will  be  discussing  this 
activity  in  more  detail  this  afternoon. 


And  now,  the  discussion  of 
provided  by  Bruce  Dodd. 


the  data  collection  will  be 


APPROACH  TO  DATA  COLLECTION  - DATA  COLLECTION 


SLIDE  D1 

The  data  collection  phase  of  the  program  was  very  care- 
fully planned  and  implemented.  Our  objective  was  to 
present  an  identical  task  to  each  of  a number  of 
separate  NDI  organizations  and  passively  measure 
technician  performance  under  their  home  base  con- 
ditions. We  wished  only  to  control  the  samples  and  the 
NDI  techniques  which  were  to  be  used  to  inspect  them. 
Since  we  were  trying  to  measure  performance  under 
locally  accustomed  conditions,  we  did  not  wish  to 
influence  individual  proficiency  nor  attempt  to  control 
the  ambient  variables  in  any  way.  The  program  required 
that  we  use  NDI  technicians  normally  assigned  to  such 
tasks  at  each  base  and  that  they  perform  under  pro- 
cedures and  conditions  that  are  routine  for  the  base. 
Ideally,  since  the  tasks  were  always  the  same,  one 
might  expect  each  technician  to  accommodate  to  the 
conditions  he  was  oriented  to  and  perform  to  the  best 
of  his  ability,  leaving  proficiency  as  the  only  real 
variable.  However,  experience  had  shown  us  that  pro- 
ficiency is  not  always  independent  of  a host  of 
ancillary  variables.  The  individual  technicians's 
reactions  to  these  variables  constitute  what  we  may 
refer  to  as  "human  factors,"  and  these  human  factors 
may  tend  to  broaden  the  measured  variations  in  per- 
formance . 

SLIDE  D2 

During  the  data  collection  phase,  we  visited  21  separ- 
ate Air  Force  installations.  All  five  Air  Logistics 
Command  bases  were  selected  for  evaluation;  then,  four 
bases  were  selected  by  the  Air  Force  Command  Monitors 
from  each  of  the  ATC , MAC,  SAC  and  TAC  commands.  Care 
was  taken  to  construct  a representative  cross-section 
of  the  command  with  the  bases  selected.  A total  of  290 
NDI  technicians  participated  in  the  program  and  ranged 
from  three  technicians  at  two  bases  to  as  many  as  47 
technicians  at  one  ALC  depot.  Considering  this  number 
of  technicians  and  the  variety  of  bases,  we  could 
reasonable  expect  the  uncontrolled  variables  to  change 
over  a wide  range  of  possibilities.  The  controlled 
variables,  on  the  other  hand,  which  included  the 
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samples,  the  NDI  standards  and  the  NDI  procedures,  were 
constant  from  base-to-base  and,  therefore,  provided  the 
uniform  base  line  required  for  direct  comparison  of 
performance  . 

SLIDE  D3 

I believe  the  best  way  to  illustrate  to  you  how  the 
data  collection  functions  were  conducted  is  to  take  you 
through  the  typical  sequence  of  activities  from  arrival 
at  an  Air  Force  facility  to  departure.  I was  the 
Lockheed  engineer  in  the  field  for  this  program  and  was 
responsible  for  its  day-to-day  conduct.  All  program 
equipment  and  the  structure  samples  were  transported 
from  base  to  base  in  a 16-foot  utility  trailer  which 
was  pulled  by  a one  ton  pick-up  truck. 

Upon  arriving  at  an  installation  the  first  concern  was 
to  acquaint  the  NDI  personnel  with  the  purposes  and 
conduct  of  the  program.  We  did  this  by  giving  program 
briefings  to  management  personnel  and  to  the  NDI 
technicians.  Both  these  briefings  were  delivered  by 
means  of  a desk  top  slide  projector  that  contained  its 
own  screen  and  a cassette  tape  player  to  make  the  oral 
presentation.  This  approach  provided  identical  brief- 
ings at  all  bases,  thus  avoiding  a potential  bias  in 
attitude  and  understanding.  The  management  briefing 
described  the  program  and  its  goals  and  included  a 
discussion  of  how  program  results  will  interact  with 
associated  engineering  technologies.  The  technician 
briefing  described  the  program  and  informed  the 
technicians  what  they  were  to  do  in  the  program  and  how 
to  complete  the  data  sheets.  Each  of  these  cassette 
briefings  are  available  here  and  are  set  up  in  the  back 
of  the  room.  If  any  of  you  are  interested  in  seeing 
these,  I will  be  happy  to  play  them  during  our  break. 

SLIDE  D4 

The  next  major  task  was  to  set-up  the  samples  and 
equipment  in  preparation  for  inspection.  The  actual 
set-up  location  and  arrangement  varied  somewhat  from 
command  to  command,  with  minor  variations  at  the  bases 
within  each  command.  Typically,  the  unloaded  trailer 
would  be  parked  in  the  x-ray  lab,  or  outside  the  x-ray 
shop  or  inside  a hangar.  The  x-ray  and  the  overhead 
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eddy  current  NDI  tasks  were  conducted  using  the  trailer 
for  a sample  mount.  The  structure  samples  and  auxiliary 
equipment  were  usually  set  up  in  the  inspection  shop  or 
in  a nearby  unused  space  where  the  lighting  and  temper- 
ature could  be  kept  within  comfortable  limits.  We 
attempted  to  make  the  location  and  ambient  conditions 
as  similar  as  practical  to  the  conditions  the  techni- 
cian was  routinely  accustomed  to.  The  program  principal 
engineer.  Bill  Sproat , was  present  with  me  at  each  base 
during  the  briefings  and  set-up  period. 

SLIDE  D5 

The  next  step  would  be  giving  assignments  to  the  NDI 
technicians  who  had  been  selected  to  participate.  Once 
the  assignments  were  made,  each  technician  was  given  a 
copy  of  the  program  NDI  manual  and  he  familiarized 
himself  with  the  particular  procedure  he  would  use  in 
his  assignment.  After  completing  his  assignment,  the 
technician  would  normally  be  assigned  another  inspec- 
tion task  as  long  as  he  continued  to  be  available. 

SLIDE  D6 

Thorough  data  collection  was  truly  the  heart  and  pur- 
pose of  the  program.  To  collect  the  data  necessary  to 
perform  an  adequate  evaluation,  we  designed  the  six 
types  of  data  collection  forms  indicated  on  this  slide. 
Three  of  these  were  to  be  filled  out  by  the  technician 
and  the  other  three  filled  out  by  the  Lockheed  field 
engineer.  These  forms  provided  for  recording  of  per- 
sonal data  from  each  technician,  equipment  parameter 
values,  flaw  indications  found  in  the  samples,  and 
information  about  the  local  facilities  available.  A 
daily  log  was  kept  to  record  environmental  data  and 
day-by-day  program  activity.  I will  now  describe  each 
of  these  forms  in  more  detail  so  that  you  will  be  more 
aware  of  the  data  that  is  now  available. 

SLIDE  D7 

TECHNICIAN  PROFILE  FORM 

The  technician  profile  form  was  completed  by  each  par- 
ticipating technician  at  the  beginning  of  the  program 
at  his  facility.  The  purpose  was  to  gather  background 


information  on  all  technicians  participating  in  the 
program.  Since  individuals  were  not  evaluated  by  name, 
their  names  were  not  recorded;  however,  numbers  were 
assigned  so  that  we  could  coordinate  other  completed 
data  forms  with  a specific  unidentified  technician.  On 
the  technician  profile  form,  the  technician  recorded 
his  assigned  ID  Number,  Job  Title  or  AFSC,  general 
educational  information,  specific  information  about  his 
NDI  training  and  experience,  job  history  in  and  outside 
the  Air  Force  and  specific  physical  data  such  as  age, 
weight,  height,  sex,  marital  status  and  physical  limi- 
tations. We  wanted  this  data  in  order  to  determine  if 
certain  types  of  backgrounds,  formal  training,  job 
experience,  physical  types,  and  so  on,  have  any  rela- 
tionship to  NDE  performance.  Such  factors,  if  shown  to 
have  valid  effects  on  NDI  reliability,  could  be  used  to 
establish  the  basis  for  an  effective  selection  or 
screening  program  for  future  NDI  personnel. 

SLIDE  D8 

EQUIPMENT  PERFORMANCE 

Specific  performance  checks  were  made  on  the  ultra- 
sonic, eddy  current,  and  x-ray  equipment  and  the 
penetrant  materials  that  were  used  at  each  base. 
Equipment  performance  data  sheets  were  filled  out  by 
the  Lockheed  engineer  to  record  the  results  of  speci- 
fied checks  on  the  NDI  equipment  and  materials.  The 
data  sheets  provided  spaces  for  completely  identifying 
the  equipment  and  materials,  equipment  settings,  test 
set-up  parameters,  process  information  and  response  or 
performance  data.  These  checks  and  resulting  informa- 
tion allowed  us  to  evaluate  to  some  degree  the  condi- 
tion of  the  equipment  at  each  base.  Generally,  we  found 
that  if  the  equipment  was  in  service  it  was  working 
satisfactorily.  Malfunctioning  equipment  was  either  out 
of  service  or  in  the  repair  shop. 

SLIDE  D9 


NONDESTRUCTIVE  INSPECTION  DATA 


Inspection 
nician  to 


data  sheets  were  provided  for 
record  specified  information 


each  tech- 
before  and 


during  each  inspection.  There  was  a separate  data  sheet 
for  each  technique,  but  similar  types  of  information 
were  requested  pertaining  to  equipment  identification, 
equipment  settings,  standards,  transducers,  probes, 
materials,  inspection  start  and  stop  times  and  tech- 
nician comments.  Note  that  this  form  did  not  request 
information  about  defects  found  in  the  sample.  The 
technician  was  instructed  to  mark  indicated  defects 
directly  on  the  structure  sample  and  to  transfer  these 
marks  to  a sample  data  sheet  following  the  inspection 
of  that  sample.  Defect  markings  were  made  on  the  sample 
using  an  erasable  marking  instrument,  and  were  com- 
pletely removed  before  the  sample  was  assigned  to  the 
next  technician. 

SLIDE  DIO 

SAMPLE  DATA  SHEETS 

These  sheets  were  actually  outline  drawings  of  the 
sample  to  be  inspected,  and  included  all  fastener  holes 
to  be  inspected.  These  sheets  were  used  to  transfer  the 
defect  markings  from  the  sample  to  the  corresponding 
location  on  the  drawing  at  the  completion  of  the  in- 
spection. Before  removing  the  marks  from  the  sample, 
the  Lockheed  engineer  visually  compared  markings  on  the 
sheet  to  those  on  the  sample  to  verify  correctness  and 
to  be  sure  that  he  understood  the  technician's  mark- 
ings. The  technician  was  instructed  to  mark  it  on  both 
sides  of  the  hole  if  there  were  cracks  on  both  sides. 

SLIDE  Dll 

FACILITY  EVALUATION  FORM 


The  facility  evaluation  form  was  used  to  record  infor- 
mation about  the  set-up  location,  ambient  conditions, 
light  and  noise  levels,  ancillary  equipment,  and  a des- 
cription of  the  inspection  area. 

DAILY  LOG 

The  Lockheed  engineer  also  maintained  daily  records 
pertaining  to  progress  on  the  program,  weather 
conditions,  special  conditions,  instrumentation 


operations  or  problems,  and  pertinent  comments  about 
the  technicians  participating. 

This  now  provides  you  with  an  overview  of  the  types  of 
data  we  collected.  Now,  I will  briefly  summarize  the 
scope  and  level  of  participation  we  experienced 
throughout  the  data  collection  phase. 

SLIDE  D12 

PARTICIPATION  LEVEL 

A summary  of  the  participation  levels  at  the  various 
commands  is  given  in  this  slide.  We  are  showing  the 
command,  number  of  technicians  that  participated  and 
the  number  of  tasks  that  they  completed.  Now  a task  is 
defined  as:  one  NDI  technique  performed  on  one  struct- 
ure sample  type.  Complete  and  partial  tasks  were  both 
counted.  We  had  a total  of  298  participating  technici- 
ans who  performed  777  total  tasks.  I might  add,  these 
technicians  inspected  a total  of  over  a half  million 
potential  flaw  sites.  On  the  average,  then,  each  tech- 
nician inspected  over  1800  sites.  Note  that  the  number 
of  participating  technicians  and  the  number  of  tasks 
completed  are  about  evenly  divided  between  the  depots 
(AFLC's)  and  the  field  bases. 

SLIDE  D13 

NDI  TECHNIQUE  USAGE 

To  give  you  an  idea  of  the  level  of  usage  each  of  the 
NDI  techniques  received  during  the  program,  this  slide 
shows  the  number  of  tasks  completed  for  each  technique. 
We  didn’t  include  the  screening  sample  tasks  (the  'G  ' 
samples)  in  this  summary.  The  automated  ultrasonic  e- 
quipment  which  will  be  described  later  today,  was  used 
at  three  bases  and  was  applied  to  only  one  type  of 
structure  sample  - thus  accounting  for  its  low  usage. 
By  contrast,  the  automated  eddy  current  bolt  hole  scan- 
ner which  will  also  be  described  in  a later  presenta- 
tion was  used  at  six  bases  and  was  applied  to  three 
structure  sample  types.  The  more  routine  ultrasonic  and 
eddy  current  techniques  received  substantial  usage  at 
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I most  all  of  the  bases.  The  penetrant  and  x-ray  proced- 

ures were  applied  to  only  one  structure  sample  type 
each  and  were  used  at  16  and  14  bases,  respectively. 
Because  of  manpower  availability,  not  all  structure 
samples  and  NDI  technique  combinations  were  evaluated 
at  all  bases. 

SLIDE  D14 

This  matrix  shows  how  the  tasks  were  concentrated  with 
respect  to  the  structure  sample  type  and  the  NDI  tech- 
nique. The  chart  indicates  how  the  plan  dedicated  spec- 
ific NDI  techniques  for  inspection  of  each  structure 
type.  The  cross-hatched  blocks  are  deliberate  omissions 
in  the  matrix. 

SLIDE  D15 

This  final  slide  shows  how  the  21  Air  Force  bases  who 
participated  in  the  program  were  distributed  throughout 
the  country.  The  color  coding  indicates  the  type  of 
command  to  which  a particular  base  belongs.  Wright- 
Patterson  AFB , although  not  a participant  in  the  data 
collection  activities,  was  the  first  base  to  be  visited 
so  that  we  could  conduct  a dry  run  of  the  program.  As 
you  see,  the  bases  were  located  in  every  geographic 
region  of  the  U.  S.  The  data  collection  phase  of  the 
program  was  carried  out  over  a period  of  2 years  and  3 
months.  The  total  distance  traveled  during  that  time 
was  about  50,000  miles. 


SLIDE  D1 


Program  Participation  Summary 

. > ■-  ' $ ■ \ 
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• n AIR  FORCE  RASES 

• VAIC  4-ATC  4-MAC  4-SAC  4-TAC 

• SEIECIED  BY  COMMAND  MONITORS 

• WISE  RANGE  OF  CONDITIONS 


• m NOI  TECHNICIANS 
• 47  MAX  J MIN 


L 


68 


SLIDE  D3 


Personnel  Brewings 

AT  EACH  VOCATION  FOR: 

• management 

• technicians  participating 


SLIDE  D4 


Program  Set  Up 


• TRAILER  LOCATED  IN  X-RAY  SHOP,  HANGAR  OR 
ADJACENT  AREA 

• SAMPLES  SET-UP  IN  NDI  SHOP  OR  UNUSED 
SHOP  AREA 
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Personnel  Assignments 


• INITIAL  TASK  ASSIGNMENT 


• REASSIGNMENT 
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Technician  Profile  Form 


Job  Title/AFSC 
Education 

NDI  Training  & Experience 
§ Job  History 
Physical  Data 
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Nondestructive  Inspection  Data 

■ NDI Technician  ID  Number 

■ Equipment/Materials  Used 
* NDi  Procedure  Identification 


■ Start /Stop  Times 
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Tasks  Completed  perNDI  Method 

NDI  METHOD TASKS* 

ULTRASONIC,  MANUAL  160  1/2* 

ULTRASONIC,  AUTOMATED  4 

EDDY  CURRENT,  SURFACE  SCAN  165  1/2* 

EDDY  CURRENT,  BOLT  HOLE,  MANUAL  253  3/4 

EDDY  CURRENT,  BOLT  HOLE,  AUTOMATED  44 

PENETRANT  63 

RADIOGRAPHIC  0<-RAY)  61 

TOTAL  751  3/4 

•EXCLUDING  SCREENING  SAMPLE  INSPECTIONS 
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Field  & Depot  Tasks  per  Sample 
& NDI  Technique 


. 1 

• 

STRUCTURE  SAMPLE 

TASKS  COMPLETED  PER  NDI  TECHNIQUE 

ULTRASONIC 

EDDY  CURRENT 

PENE- 

TRANT 

RADIO- 

GRAPHIC 

MANUAL 

AUTO- 

MATIC 

SURFACE 

SCAN 

B.H.  - 

MAN 

B.H.  ■ 
AUTO 

A - C-130  WING  BOX 

53 

4 

63  1/2 

B - C-130  WING  BOX  SEGMENTS 

102 

61 

C - WING  RISERS 

32 

63 

D - KC-135  WING  SECTIONS 

46  V2 

71  1/2 

9 

E - F-104  WING  FITTINGS 

106  1/2 

21  1/2 

F - C'5  BOX  BEAM 

29 

76  3/4 

13  1/2 

G - SCREENING  FLAT  PLATES 

9 1/4 

16 

TOTALS 

169  3/4 

4 

181  1/2 

253  3/4 

44 

63 

fl 

74 


L 


SLIDE  El 

TEARDOWN  RESULTS 

So  far,  we  have  seen  what  the  structures  looked  like 
and  how  the  data  acquisition  progressed.  Now,  the  dis- 
cussion will  center  around  the  measurements  of  the 
flaws  by  destructive  methods. 

Following  the  completion  of  the  Phase  II,  Data  Collect- 
ion, a teardown  examination  was  made  of  the  program 
samples  to  accurately  determine  and  fully  characterize 
their  defect  content.  In  the  planning  phase,  we  init- 
ially catalogued  all  known  defects  and  estimated  their 
uncovered  lengths  using  careful  visual  and  eddy  current 
examination.  In  order  to  provide  an  accurate  evaluation 
of  Air  Force  NDI  capability  based  on  data  collection 
from  these  samples,  it  was  necessary  to  acquire  com- 
plete and  accurate  information  about  the  flaw  sites. 

In  addition  to  the  initial  information,  the  teardown 
inspection  provided  (1)  knowledge  of  additional  flaws, 
unknown  in  the  planning  phase,  for  a complete  account- 
ing of  all  cracks,  (2)  confirmation  of  the  existence  of 
flaws  at  locations  which  exhibited  a high  incidence  of 
"false  calls,"  and  elimination  as  suspect,  those  sites 
which  exhibited  NDI  indications  but  contained  no  actual 
flaw,  and  (3)  a direct  accurate  measurement  of  the  geo- 
metry and  orientation  of  identified  flaws. 

SLIDE  E2 
SCOPE 

All  structure  samples,  except  for  type  C and  G struct- 
ure samples,  were  included  in  the  teardown  inspection. 
Since  the  characteristics  of  service-induced  flaws  vary 
widely,  the  complete  parameters  cannot  be  defined  from 
surface  length  measurements  alone.  Since  the  actual 
structure  samples  contained  natural  flaws,  it  was 
therefore  necessary  to  subject  them  to  the  teardown 
inspection.  The  type  C and  G samples,  on  the  other 
hand,  contained  flaws  which  were  machine  induced  under 
closely  controlled  conditions.  The  size  and  shape  of 
those  flaws  were  therefore  known  with  sufficient  detail 
for  analysis  without  further  examination. 
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INITIAL  PREPARATION 

The  teardown  examination  can  be  separated  into  two  act- 
ivities, which  were  the  initial  preparation  and  examin- 
ations and  the  destructive  examinations.  The  initial 
stage  included  the  sequence  detailed  on  this  chart.  In 
general,  the  sequence  included  disassembly  of  the  sam- 
ple, stripping  of  surface  finish,  several  steps  of 
cleaning  and  etching,  visual  and  penetrant  inspections 
to  determine  all  suspect  flaw  sites,  marker  identifica- 
tion of  each  suspect  area,  sectioning  layout  on  the 
part,  photographing  the  part,  sectioning  them  into  in- 
dividual laboratory  specimens  containing  the  suspect 
flaw,  and  additional  cleaning  with  a mild  alkaline  sol- 
ution followed  by  an  ultrasonic  cleaning. 

Structure  samples  A,  B,  E and  F were  totally  disassem- 
bled. The  sample  A portion  was  cut  from  the  intact  wing 
box,  rivets  sheared  off  at  the  buck  tail  and  skins  re- 
moved from  stringers.  Sample  B was  treated  in  the  same 
manner  but  no  initial  cutting  was  required.  Sample  D 
required  no  disassembly  and  sample  E breakdown  was  ach- 
ieved by  unbolting  the  cracked  fittings  from  the  dummy 
cover  plates.  Sample  F sections  were  cut  from  the  total 
box  beam  and  separated  into  spar  caps,  webs  and  web 
stiffeners.  Paint  was  removed  from  all  structure  samp- 
les with  a paint  stripper.  After  stripping,  the  samples 
were  cleaned  in  an  alkaline  solution,  water  rinsed, 
dipped  in  an  acid  neutralizing  bath,  water  flushed  and 
oven  dried. 

Fluorescent  penetrant  inspections  were  performed  on  the 
samples  by  QA  laboratory  personnel.  Flaw  indications 
were  identified  on  the  pieces  at  all  suspect  locations 
using  a red  marker.  Samples  A and  B were  also  inspected 
further  with  a special  automatic  eddy  current  surface 
scanning  device,  developed  at  Sacramento  ALC.  The  re- 
sults of  the  penetrant  and  special  eddy  current  scans, 
along  with  high  incidence  of  "false  calls"  derived  from 
the  field  data,  were  added  to  the  list  of  flaw  suspect 
locations . 
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Each  fastener  hole  containing  a suspect  flaw  was  iden- 
tified with  an  engraved  ID  number  and  the  clock  orien- 
tation was  punch-marked  near  the  hole.  After  the  sample 
pieces  were  marked  for  cutting  and  photographed,  they 
were  cut  into  smaller  laboratory  specimens  containing 
one  or  more  flawed  fastener  holes.  Those  specimens  with 
a single  hole  measured  1 to  2 inches  square.  The  larger 
specimens  contained  large  flaws  or  more  than  one  hole. 
The  specimens  were  thoroughly  cleaned  using  a mild  alk- 
aline solution  followed  by  immersion  in  an  ultrasonic 
cleaning  tank  containing  acetone.  At  this  point,  the 
specimens  were  examined  optically  to  determine  the  need 
for  further  processing. 

SLIDE  E4 

DESTRUCTIVE  INSPECTION 

Those  specimens  whose  flaws  were  obvious  visually  or 
when  viewed  with  a binocular  microscope  having  up  to 
30X  magnification  were  opened  for  crack  characteriza- 
tion without  further  processing.  Otherwise,  additional 
processing  was  necessary  to  determine  the  precise  loca- 
tion of  the  flaw  prior  to  opening  the  specimen.  Pro- 
cessing now  depended  on  whether  the  specimens  still  had 
an  anodize  layer  at  this  point.  Specimens  from  samples 
A,  B and  F had  an  anodize  layer  which  now  required 
stripping  in  an  etch  solution.  Afterwards,  an  optical 
examination  was  made  to  cull  the  visually  detectable 
cracks,  which  were  opened  without  further  processing. 
If  a crack  was  not  detected,  the  specimen  was  alkaline 
etched  and  deformed  under  tension  if  necessary  until 
all  flaws  were  identified  and  opened.  Samples  D and  E 
had  been  previously  etched  prior  to  incorporation  in 
the  program  to  remove  the  anodize  layer.  Specimens  from 
these  were  inspected  using  an  eddy  current  bolt  hole 
probe  to  pinpoint  flaw  indication.  Then  additional 
etching  and  optical  examination,  sometimes  aided  by 
flourescent  penetrant,  were  provided  as  necessary  to 
determine  where  the  specimen  should  be  opened.  All  the 
specimens  were  opened  at  suspect  flaw  sites.  In  some 
cases,  the  previous  flaw  indications  were  due  to  gouges 
and  pitting,  as  in  the  case  of  sample  D specimens,  or 
to  generally  poor  hole  wall  conditions.  The  presence  of 
these  noncrack  flaws  explains  at  least  some  of  the 
"false  calls"  reported  during  the  program. 


Flaw  size  measurement  on  opened  cracks  larger  than  .010 
inches  were  made  with  the  use  of  a metal  scale  and  a 
30X  magnification  binocular  microscope.  Precision  of 
the  optical  measurements  was  +0.005  with  recorded  val- 
ues rounded  to  the  nearest  0.l5  inch.  For  flaws  smaller 
than  .010  inches,  measurements  were  made  using  a grati- 
cule on  a scanning  electron  microscope  display.  The 
precision  was  +0.0005  inches,  rounded  to  the  nearest 
0.001  inch.  Flaw  plane  angularity  with  respect  to  the 
flaw  axis  and  the  true  radial  direction  were  measured 
using  a protractor  overlay. 
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FLAW  CATEGORIES 


Ca  tegor ies 
angularity 
here.  Flaw 
cation  and 


for  recording  flaw  types,  dimensions,  and 
pertinent  to  the  program  specimens  are  shown 
types  were  categorized  according  to  flaw  lo- 
conf ig uration  within  the  sample.  Flaw  dimen- 


sions were  measured  axially  along  the  hole  wall  and  ra- 
dially from  the  hole  edge.  Dimensions  and  crack  type 
were  recorded.  An  angle  A (alpha)  was  recorded  for 


flaws  whose  axial  dimension  varied  from  the  true  axial 
direction.  An  angle  B (beta)  was  recorded  for  flaws 
whose  radial  dimension  varied  from  the  true  radial 


direction . 
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SUMMARY  OF  TEARDOWN  RESULTS 

A catalogue  of  accurate  crack  dimensions  was  construct- 
ed from  the  teardown  inspection  data.  These  data  were 
then  available  for  use  in  analyzing  the  program  re- 
sults. Since  complete  flaw  dimensions  and  shape  infor- 
mation is  available,  cracks  can  be  stated  in  terms  of 
flaw  area,  flaw  length  or  flaw  aspect  ratio.  This  was 
accomplished  in  order  that  flaw  characteristics  could 
be  evaluated  to  determine  their  influence  on  detection 
and  all  "false  call"  sites  could  be  resolved.  In  the 
case  of  sample  D,  flaws  that  were  previously  regarded 
as  very  small  cracks  were  determined  instead  to  be  gou- 
ges and  pitting,  as  Bill  Lewis  previously  noted.  Sample 
D,  therefore,  was  eliminated  from  further  analysis  in 
evaluating  the  program  results. 
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Much  the  same  thing  was  experienced  with  the  sample  E 
specimens  wherein  only  seven  of  the  27  initially  sus- 
pected and  catalogued  flaws  were  verified  as  true 
cracks  - the  remaining  suspects  were  due  to  poor  hole 
surface  conditions.  The  teardown  inspection  resulted  in 
a revision  in  the  number  of  cracks  for  some  sample 
types;  fifty  small  cracks  above  the  number  initially 
recorded,  were  discovered  in  sample  A;  in  sample  F,  34 
of  the  43  originally  catalogued  cracks  were  verified  as 
true  cracks.  The  crack  size  estimates  for  samples  A and 
B did  not  change  dramatically.  Actually,  the  initially 
estimated  flaw  sizes  for  all  true  cracks  in  samples  A, 
B,  E and  F were  reasonable  close  to  the  actual  sizes. 
Five  inspection  sites  that  had  received  a high  inciden- 
ce of  "false  calls”  during  the  program  were  examined 
and  all  were  found  to  have  actual  cracks.  These  were 
added  to  the  list  of  cracks  and  were  included  in  the 
subsequent  analysis  update.  The  overall  reliability  re- 
sults obtained  on  these  structure  samples,  after  the 
flaw  catalog  revision,  will  be  presented  by  Bill  Lewis. 
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Purpose  of  Teardown  Inspection 


• DETERMINE  TRUE  TOTAL  Of  CRACKS 

• INVESTIGATE  TALSE  CALL’  SITES 

• CHARACTERIZE  CRACKS 

TRUE  SIZE 
LOCATION 
ORIENTATION 
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• SAMPLES  A.  8.  0,  E AND  f «RE  INCLUOED  BECAUSE 
OF  UNKNOWN  CRACK  CHARACTERISTICS 

• SAMPLES  C AND  G WERE  EXCLUDED  BECAUSE  CRACK 
CHARACTERISTICS  WERE  KNOWN 
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Tear  down  Inspection,  Final  Sequence 


SLIDE  E3 


Teardown  Inspection,  Initial  Sequence 

REMOVE  TEST  SECTION  FROM  STRUCTURE 

▼ 

SURFACE  PREPARATION  AND  PENETRANT  INSPECT 

▼ 

RECORD  SUSPECT  FLAWS 

▼ 

LAYOUT  ANO  IDENTIFY  FOR  SECTIONING,  PHOTOGRAPH 

▼ 

SECTION  INTO  LABORATORY  SPECIMENS  AND  CLEAN 
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» STRIP  ANODIZE 
► MICROSCOPIC  EXAM 

♦ OPEN  CRACKS 

♦ ETCH  (REPEAT  i & 31 

♦ DEFORM  IN  TENSION 
(REPEAT  Z & 31 

♦ CONFIRM  PRE-EXISTENCE 
OF  CRACKS 

♦ MEASURE  CRACKS 


♦ EDDY  CURRENT  BH 
INSPECT 

♦ OPEN  AT  SUSPECT 
LOCATIONS 

♦ REPORT  NO  CRACKS 


♦ EDDY  CURRENT  BH 
INSPECT 

♦ ETCH 

♦ MICROSCOPIC  EXAM 

♦ OPEN  CRACKS 

♦ PENETRANT  INSPECT 

♦ MEASURE  CRACKS 


Flaw  Location  at  Fastaner  Holes 

RAW  LOCATION  AT  FASTENER  HOIS 


r ■ RADIAL 
b • AXIAL  LENGTH 


Flaw  Categories  and  Meaurement  Criteria 


FLAW  ORIENTATION  AT  FASTENER  HOLES 


NOTE: 

a AND  P ARE  MEASURED 
FROM  FLAW  ORIGIN 
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Summary  of  Teardown  Inspection 

• CATALOGUE  OF  ACCURATE  DIMENSIONS  AND  CHARACTERISTICS 
FOR  All  FLAWS 

• FIVE  'YALSE  CALL"  SITES  WERE  RESOLVED  AS  ACTUAL  CRACKS 

• SOME  PREVIOUS  FLAW  SITES  ELIMINATED  (SAMPLE  Dl 


OVERALL  PROGRAM  RESULTS 


SLIDE  FI 

I'm  going  to  be  showing  you  a series  of  slides  next 
that  display  graphically  the  overall  flaw  detection 
capabilities  that  we  observed  during  the  program.  Be- 
fore we  look  at  these  slides  however,  it  is  extremely 
important  to  point  out  that  generalizations  cannot  and 
must  not  be  made  from  these  overall  curves.  They  repre- 
sent the  total  flaw  detection  probabilities  of  all  the 
298  individual  participants  in  the  program  measured  un- 
der very  specific  conditions.  We  do  not  possess  the 
knowledge  or  the  experience  that  would  allow  us  to 
translate  these  flaw  detection  results  to  any  other 
specific  set  of  inspection  conditions.  We  do  know,  and 
these  overall  results  show  us,  that  a change  in  any  of 
the  parameters  associated  with  a specific  nondestruc- 
tive inspection  can  have  a dramatic  effect  on  flaw  de- 
tection probability. 

These  curves  do  represent  the  overall  crack  detection 
capability  of  all  the  NDI  technicians  included  in  the 
program  as  measured  under  the  specific  sets  of  condi- 
tions noted.  Statistically,  they  represent  total  Air 
Force  capability  at  the  present  time  under  the  same 
conditions,  and  they  should  be  viewed  in  that  context. 

The  curves  are  a regression  fit  to  individual  data 
points  established  by  a large  number  of  independent  at- 
tempts to  find  each  individual  flaw.  For  each  set  of 
conditions,  we  have  plotted  the  mean  crack  detection 
probability  of  all  technicians  vs  crack  size  along  with 
the  lower  95%  confidence  bound  to  indicate  the  combined 
effect  of  data  scatter  and  sample  size  on  the  statisti- 
cal certainty  of  the  measurements. 

We  have  also  included  on  each  of  these,  a plot  of  the 
crack  detection  probability  for  the  top  10%  of  the 
technicians  included  in  the  program  which  dramatically 
shows  the  potential  that  can  be  obtained. 

These  various  conditions  are  shown  in  the  following 
si  ides  . 
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PROGRAM  ADDITIONS 


SLIDE  G1 

Approximately  half  way  through  the  data  acquisition 
phase  of  our  preliminary  data,  it  was  decided  that  this 
would  be  a golden  opportunity  to  evaluate  several  re- 
cent improvements  made  in  nondestructive  testing  meth- 
ods. Specifically,  these  consisted  of  the  automated  ed- 
dy current  bolt  hole  inspection  equipment  originally 
developed  by  the  Sacramento  Air  Logistics  Center  and 
commercially  marketed  by  Gulton  Industries  and  also 
automated  ultrasonic  inspection  equipment  specifically 
designed  for  the  inspection  of  the  outer  layer  with  the 
fastener  installed.  This  equipment  was  originally  deve- 
loped by  the  Boeing  Company  under  contract  with  the  Air 
Force  Materials  Laboratory.  The  Gulton  eddy  current 
bolt  hole  equipment  was  very  new  at  this  point  in  time 
and  was  just  beginning  to  be  phased  into  the  Air  Force 
inventory,  therefore  we  carried  it  with  us.  In  order  to 
acquire  data  on  its  use  and  a measure  of  the  resulting 
improvement  in  detection  reliability  accorded  by  such 
equipment,  it  was  decided  to  incorporate  it  as  soon  as 
possible  in  the  Reliability  Measurement  Program. 

SLIDE  G2 

On  the  the  other  hand,  the  automated  ultrasonic 
roto-scan  equipment  existed  only  in  various  prototype 
stages  and  a version  of  the  prototype  equipment  was 
included  in  our  program  in  order  for  us  to  get  a 
similar  cursory  evaluation  of  its  crack  detection 
capabil ity. 


SLIDE  G3 

Besides  these  two  additions,  a third  addition  to  the 
program  was  made  during  this  period  which  was  funded  by 
the  Air  Force  Materials  Laboratory  and  consisted  of 
some  flat  plates  with  fatigue  cracks  originating  from 
the  fastener  holes  in  these  plates.  The  purpose  of  this 
addition  was  an  attempt  at  determining  if  a measure  of 
technician  proficiency  in  detecting  cracks  could  be 
obtained  from  inspections  made  on  a few  simple  repro- 
ducible flat  plates  instead  of  the  much  more  expensive 
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PRECEDING  PACT  BLaJCC-NOT  FILMED 


unr eproducible  and  time  consuming  complex  structure.  If 
such  a correlation  between  the  flat  plates  and  the 
complex  structure  exists,  then  the  results  are  obvious. 
These  flat  plates  added  to  the  program  became  known  as 
the  technician  screening  samples.  We  plan  to  cover  each 
one  of  these  additions  to  our  basic  program  in  more 
detail  because  we  feel  that  each  provided  us  with  a 
valuable  insight  and  important  results  that  can  be 
applied  to  other  areas  for  improvement  purposes.  Bernie 
Boisvert  from  the  San  Antonio  Air  Logistics  Center  will 
now  briefly  describe  for  you  the  automated  eddy  current 
bolt  hole  inspection  equipment  and  the  results  we  ob- 
tained from  using  it. 
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PROGRAM  ADDITIONS  - AUTOMATIC  EDDY  CURRENT 


SLIDE  HI 

The  eddy  current  inspection  method  is  currently  accept- 
ed as  one  of  the  most  potentially  reliable  and  repeat- 
able  methods  in  the  aircraft  NDI  repertoire.  It  is 
simple  to  apply  and  often  requires  less  surface  pre- 
paration than  other  comparable  methods.  It  usually  does 
not  require  a precise  prediction  of  where  the  flaw  will 
occur  as  does  the  ultrasonic  method. 

The  eddy  current  method  can  be  applied  in  various  ways, 
which  include  both  hand  scanning  and  automated  scan- 
ning. It  can  be  applied  to  planar  or  contour  surfaces 
and  to  small-diameter  fastener  holes. 

Today,  I want  to  discuss  an  application  of  the  method 
to  the  inspection  of  fastener  holes,  for  this  is  a 
critical  application  for  which  significant  advances  in 
eddy  current  instrumentation  have  come  about  in  recent 
years . 

SLIDE  H2 

One  disadvantage  of  the  eddy  current  method  for  bolt 
hole  inspection  has  been  the  lack  of  repeatability  and 
reliability  of  inspection  results  when  hand  scanning 
techniques  are  used.  Hand  scanning  is  highly  operator 
dependent  and  results  can  vary  considerably  from  one 
inspector  to  another.  This  is  because  the  quality  of 
hand  scanning  depends  on  the  operator's  skill,  his 
understanding  of  the  specific  problem,  his  physiolog- 
ical motor  functions  and  additional  factors  that  can  be 
highly  variable  and  unpredictable. 


There  was  a need  to  remove  as  much  of  the  operator- 
dependency  as  possible,  in  order  to  exploit  more  of  the 
method's  inherent  sensitivity  and  reliability.  This 
could  be  done  by  making  certain  changes  to  eddy  current 
equipment  involving  an  automated  scanning  head  and  a 
way  of  presenting  the  inspection  information  that  would 
be  less  dependent  on  "transient  eyeball  judgement." 
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SLIDE  H3 


In  1970,  an  automated  eddy  current  bolt  hole  system  was 
developed  at  McClellan  Air  Force  Base,  California,  re- 
sulting in  the  prototype  shown  here,  which  was  used  on 
the  F-104  aircraft. 

The  system  consisted  of  a motorized  scanner  to  hold, 
rotate, and  spirally  index  the  bolt  hole  probe  into  and 
through  the  bolthole.  A Magnaflux  ED-520  was  used  as 
the  electronic  heart  of  the  system.  A strip  chart 
recorder  provided  a permanent  record  of  the  indications 
as  produced  in  the  ED-520  plus  a rotation  marker.  The 
scanner  and  recorder  were  controlled  by  switches  on  the 
control  panel.  Various  hardware  was  included  for  cali- 
brating the  system  and  rigidly  mounting  the  scanner  to 
the  part  being  inspected,  which,  in  the  initial  case, 
was  the  F-104  wing.  The  entire  system  was  transported 
and  used  in  a single  case. 

SLIDE  H4 

This  system  was  somewhat  cumbersome  in  its  overall  size 
and  in  the  necessity  to  mount  the  scanner  on  the  part 
being  inspected.  Also,  because  it  was  initially 
dedicated  to  the  F-104  problem,  it  had  some  minor 
limitations  relative  to  other  applications.  Hence,  it 
was  not  suitable  for  general  widespread  field  use. 

SLIDE  H5 

During  the  next  several  years,  simultaneous  improve- 
ments were  made  by  various  individuals  to  produce  a 
hand-held  portable  system  for  field  inspections.  A 
meeting  was  held  at  Kelly  AFB  in  February  1974  to 
evaluate  three  different  systems  developed  by  three 
separate  organizations:  Gulton  Industries,  the 
Loc kheed-Geo rg ia  Company,  and  Oklahoma  City  Air 
Logistics  Center  (ALC) . The  Gulton  Industries  FD-222, 
which  was  evaluated,  is  shown  in  this  slide. 

SLIDE  H6 


The  Lockheed  version  that  was  presented  for  evaluation 
at  that  time  is  shown  here. 


SLIDE  H7 

The  model  FD-100,  made  by  Gulton  Industries,  now  cur- 
rently in  use  at  several  Air  Force  installations, 
resulted  from  the  recommendations  and  specifications  at 
this  meeting.  The  use  of  the  Gulton  FD-100  currently 
appears  in  the  AFTO-36  NDI  manuals  for  various  air- 
craft. The  FD-100,  as  you  can  see  here,  has  essentially 
the  same  basic  elements  as  the  original  McClelland 
prototype,  but  is  considerably  moreunitized  and  port- 
able . 


SLIDE  H8 


The  automated  probe  head  of  the  FD-100  does  not  require 
mounting  to  the  part  being  inspected.  It  is  hand-held 
against  the  part  while  the  probe's  spiral  rotation  into 
or  out  of  the  fastener  hole  is  conveniently  controlled 
by  a rocker  switch  on  the  probe  head.  Probes  are  inter- 
changeable and  are  made  especially  for  the  unit  with 
common  threaded  connectors. 


SLIDE  H9 

The  FD-100  is  convenient  for  field  use.  Its  size  and 
weight  make  it  easily  portable  and  useful  under  a wing 
or  other  space-restricted  area.  The  fact  that  a record- 
ing is  made  of  the  eddy  current  indications  eliminates 
the  need  for  the  inspector  to  try  to  keep  one  eye  on 
the  probe  and  the  other  eye  on  the  meter  - now  he  can 
observe  the  probe  and  read  the  recording  later.  If  he 
is  not  satisfied  with  the  quality  of  the  indications, 
he  can  make  the  changes  he  thinks  necessary  and  repeat 
the  hole-scan.  Recordings  of  each  hole  he  "inspects," 
including  the  standard,  can  be  kept  in  the  permanent 
files.  Obviously,  great  improvements  in  reliability  and 
data  analysis  are  inherent  in  this  type  of  system. 


The  automated  eddy  current  bolt  hole  system  has  not 
replaced  the  manual  scan  system  - both  are  now  used  in 
the  Air  Force  and  commercial  aircraft  maintenance. 
Therefore,  it  was  highly  desirable  to  incorporate  an 
evaluation  of  the  FD-100  system  into  the  ongoing  NDI 
Reliability  Measurement  Program,  where  the  results 
could  be  compared  directly  with  the  manual  scan 


101 


results.  The  automated  system  was  not  initially 
included  in  the  "Have  Cracks"  program  because  the 
system  was  not  generally  available  at  Air  Force 
installations  . 

SLIDE  H10 

Two  FD-100  systems  were  purchased  and  they  were 
incorporated  into  the  program  at  Dover  AFB  in  December 
1976,  and  used  throughout  the  remainder.  Lockheed 
supplied  step-by-step  self-training  procedures,  pract- 
ice samples  and  inspection  procedures  for  using  the 
system  on  the  program  samples.  The  s e 1 f- t r a i n i ng 
procedures  and  practice  samples,  shown  here,  offered  a 
means  to  familiarize  the  Air  Force  technicians  with  the 
operation  and  use  of  the  FD-100  prior  to  applying  it  to 
the  structure  samples,  since  most  technicians  would  not 
have  experience  in  using  it  previously. 

SLIDE  Hll 

The  FD-100  automated  bolt  hole  system  was  used  on 
structure  type  D;  shown  here,  and  

SLIDE  H12 


structure  type  F (shown  here).  Since,  manual  bolt  hole 
inspections  were  also  made  on  these  structure  types.  We 
can  compare  the  results  of  both  types  of  eddy  current 
inspections  on  the  same  type  structure  samples. 


SLIDE  H13 


This  slide  shows  the  results  for  structure  sample  E in 
terms  of  the  mean  and  95%  confidence  curves.  In  addi- 
tion to  a greater  overall  percentage  of  detection,  we 
see  a noticeable  improvement  in  detection  of  the  cracks 
below  0.20"  axial  length.  The  "manual"  data  were 


accumulated  from 
■automatic"  data 
included  105-1/2 
ively . 


all  21  of  the  Air  Force  bases  and  the 
were  collected  from  9 bases.  These 
and  21-1/2  inspection  tasks,  respect- 
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SLIDE  H14 


Next  is  the  slide  showing  the  "manual"  and  "automatic" 
results  for  structure  type  F.  The  improvement  in 
inspection  reliability  is  dramatic  for  this  application 
for  all  crack  sizes  in  the  sample.  The  "manual"  data 
for  sample  F were  collected  from  19  bases  in  76-3/4 
inspection  tasks  and  the  "automatic"  data  were  col- 
lected from  7 bases  in  13-1/2  inspection  tasks. 

SLIDE  H15 

Results  from  the  "Have  Cracks"  program  strongly  indi- 
cate that  the  automated  eddy  current  bolt  hole  system 
does  indeed  increase  the  potential  for  higher  inspect- 
ion reliability  for  fastener  holes.  I believe  a fair 
comparison  between  manual  and  automated  units  has  been 
made  in  the  two  applications  just  shown.  This  final 
slide  summarizes  the  factors  for  which  improvements 
have  been  required  for  the  FD-100  and  for  development 
of  future  automated  systems.  Modifications  to  the  basic 
system  are  continually  being  made  to  bring  about  incre- 
mental advancements  in  these  factors  to  improve 
operating  characteristics  and  crack  detection 
capabi 1 i ties . 
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PROGRAM  ADDITIONS  - ULTRASONIC  "ROTO-SCANNER" 


SLIDE  1-1 

As  you  have  seen  from  the  previous  presentation,  once 
the  fastener  has  been  removed  from  the  fastener  hole, 
eddy  current  bolt  hole  inspection  provides  us  with  a 
very  good  practical  method  for  detection  of  fatigue 
cracks.  It  is  however,  often  undesirable  or  even 
impossible  to  remove  fasteners  in  order  to  gain  access 
for  an  eddy  current  bolt  hole  inspection.  This  is 
especially  true  where  interference  fasteners  are 
concerned.  The  removal  and  replacement  of  interference 
fasteners  are  not  always  easy  and  the  risk  is  ever 
present  that  damage  to  the  hole  or  the  fastener  may 
result.  Therefore  the  need  has  existed  for  some  time 
for  a practical  means  of  inspecting  fastener  holes 
without  fastener  removal. 

A number  of  interesting  approaches  have  been  proposed 
and  investigated  to  varying  extents  including  acoustic 
impact,  ultrasonic,  and  both  low  frequency  and  multi- 
-frequency  eddy  current.  The  acoustic  impact  technique 
was  pursued  during  the  late  60's  and  proved  to  be 
highly  sensitive  to  a wide  range  of  variables  inherent 
in  aircraft  construction  which  inhibited  fatigue  crack 
detection.  Both  the  low  frequency  and  the  multi- 
frequency eddy  current  approaches  are  still  under 
development  to  improve  resolution  although  several  low 
frequency  eddy  current  instruments  are  already  on  the 
market  and  being  used  in  some  applications . 

The  ultrasonic  approach  is  limited  to  positive  interro- 
gation of  the  outer  layer  of  a joint  as  the  energy  will 
not  always  penetrate  the  faying  surface  to  subsequent 
layers.  In  spite  of  this  limitation,  however,  ultra- 
sonics does  have  the  potential  for  detecting  very  small 
cracks  under  countersunk  fastener  heads,  along  the  hole 
wall,  and  at  the  faying  surface  of  the  outer  layer. 
Sometimes  it's  possible  to  penetrate  through  to  the 
inner  layer  and  detect  cracks  there  also.  It  was  for 
these  reasons  that  the  Air  Force  Materials  Laboratory 
actively  pursued  the  development  of  a semi-automatic 
ultrasonic  approach  for  the  detection  of  cracks  under 
fastener  heads. 
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SLIDE  1-2 


Under  contract  from  the  Materials  Laboratory,  Boeing 
developed  a working  model  of  such  a device  in  1975. 
This  equipment,  which  later  became  known  as  the  Boeing 
■ Roto-Scanner " , as  shown  in  this  slide  is  being  eval- 
uated by  Boeing  engineers  on  the  spanwise  splices  of 
the  C-5A  full  scale  fatigue  test  article. 

SLIDE  1-3 

The  Roto-Scanner  had  several  very  important  advantages. 
It  provided  a highly  stable  and  accurate  means  of  posi- 
tioning and  rotating  the  transducers  around  the  fasten- 
er head.  The  transducers  were  extremely  accurate  and 
the  boots  or  shoes  used  to  couple  energy  into  the  work 
piece  was  very  efficient  and  allowed  for  smooth  trans- 
ducer rotation  with  no  accompanying  couplant  problems. 
Manual  manipulation  was  eliminated,  providing  a smooth, 
continuous  scan  around  the  fastener  head  circumference. 

There  were  some  distinct  limitations  to  this  early 
equipment,  however,  and  some  improvement  was  obvious. 

SLIDE  1-4 

Four  more  of  these  early  prototype  devices  were  built 
by  Boeing  with  some  improvements  for  evaluation  pur- 
poses. As  a result  of  these  evaluations,  several  diff- 
erent organizations  began  making  their  own  improvements 
to  these  units  or  building  their  own  version  of  the 
Roto-Scanner.  Consequently,  there  are  a number  of 
slightly  different  Roto-Scanners  at  various  locations 
around  the  country  today,  most  of  them  slightly  differ- 
ent and  unique  from  all  others. 

During  the  time  when  everyone  was  making  their  own 
modifications  to  the  Roto-Scanner  concept,  it  was 
determined  advisable  to  include  a Roto-Scanner  into  the 
on-going  NDI  reliability  program  in  order  to  obtain  a 
measure  of  its  effect  on  inspection  reliability.  The 
particular  edition  of  the  Roto-Scanner  selected  for 
inclusion  in  the  program  is  shown  in  this  slide,  and 
was  built  by  the  San  Antonio  Air  Logistics  Center.  It 
features  a hand  rotating  knob  which  replaced  the  motor- 
ized mechanism  of  the  earlier  models. 


SLIDE  1-5 


The  precise  position  of  the  transducers  was  controlled 
by  using  templates  as  shown  in  this  slide  in  order  to 
accurately  direct  the  narrow  beam  of  sound  energy  to 
the  desired  location  depending  on  the  size  fastener 
hole  and  sheet  thickness  being  inspected. 

Scans  around  fasteners  were  made  using  one  transducer 
at  a time  with  one  full  clockwise  rotation  on  one 
transducer  and  one  full  counterclockwise  rotation  on 
the  other.  A UM-775  Ref lectoscope  with  a Transigate 
alarm  was  used  as  instrumentation  for  the  flaw  detect- 
ion. Echo  signals  occuring  within  a narrow  gate  set-up 
on  a reference  standard,  triggered  the  alarm  at  50%  of 
full  screen  height. 

SLIDE  1-6 

Since  the  ultrasonic  scanner  was  relatively  new,  a 
self-training  package  was  provided  for  the  technicians 
to  gain  proficiency  with  the  equipment  and  procedures. 
This  package  contained  step-by-step  instructions  for 
set-up  and  operation  along  with  pieces  of  structure 
containing  identified  flaws.  Illustrations  of  the 
instrument  response  on  these  flaws  allowed  the  operator 
to  determine  how  correct  operation  was  obtained.  After 
a few  days  of  training  and  familiarization  with  the 
equipment  on  structure  samples  having  real  cracks,  the 
technician  was  ready  to  test  his  proficiency  on  the 
program  samples. 

SLIDE  1-7 

Shown  here  is  the  improvement  obtained  by  using  the 
Roto-Scanner  compared  to  hand  scanning  on  Specimen  A. 

A total  of  4 participants  at  3 bases  contributed  to  the 
data  used  to  draw  the  curve.  There  can  be  no  statisti- 
cal significance  attached  to  this  because  of  the  small 
sample  size;  however,  the  potential  for  improvement  is 
dramatic . 
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PROGRAM  ADDITIONS  - TECHNICIAN  SCREENING  SAMPLES 

SLIDE  J-l 

A preliminary  evaluation  of  overall  Air  Force  NDI 
capability  was  made  after  data  had  been  collected  from 
the  first  ten  Air  Force  bases  in  the  NDI  Reliability 
Measurement  Program.  This  evaluation  tended  to  verify 
an  earlier  assumption  that  the  proficiency  of  Air  Force 
NDI  technicians  would  vary  over  a wide  range.  Further- 
more, the  performance  of  a few  individual  technicians 
on  the  samples  in  the  NDI  program  was  quite  poor,  while 
performances  of  some  others  were  outstanding  in  compar- 
ison. 


It  was  feared  that  the  same  range  of 
experienced  when  technicians  conduct 
flying  aircraft.  Of  course,  we  always 
the  best  performance  possible  whenever 
conducted  on  actual  aircraft  hardware. 


proficiency  is 
inspections  on 
desire  to  have 
inspections  are 


The  idea  evolved  that  perhaps  the  proficiency  of  NDI 
technicians  could  be  determined  through  the  use  of 
screening  samples  containing  real  fatigue  cracks.  Tech- 
nicians would  be  subjected  to  tests  wherein  they  would 
inspect  a very  large  number  of  sites,  some  of  which 
would  contain  actual  fatigue  cracks.  An  analysis  would 
be  made  of  the  results  of  individual  technicians' 
performance  to  provide  a picture  of  their  inspection 
reliability.  These  analyses  would  be  compared  to  the 
same  technicians'  performance  on  the  program  structure 
samples  to  determine  the  validity  of  judgements  made 
based  on  the  predictions  of  simple  flat  plates.  Subse- 
quently, an  experimental  screening  program  was  designed 
by  Lockheed  and  integrated  into  the  "Have  Cracks"  pro- 
gram. The  screening  program  was  funded  by  the  Air  Force 
Materials  Laboratory  and  implemented  through  the  Air 
Force  Logistics  Command  program  with  Lockheed. 

I must  point  out  that  no  one  was  certain  that  such  a 
screening  program  would  provide  valid  results  for  the 
purpose  intended.  It  was  completely  experimental.  We 
wanted  to  determine  if  it  were  possible  to  measure  NDI 
capability  using  flat  panels  and  test  samples  rather 
than  actual  aircraft  structures.  It  if  proved  to  be 
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possible,  I think  it  is  clear  what  the  subsequent 
benefits  would  be  in  training  and  testing  NDI  techni- 
cians and  in  selection  of  high  performance  technicians 
to  conduct  critical  inspections.  The  screen'ng  program 
was  designed  to  help  us  determine  that  possibility. 

SLIDE  J-2 

The  screening  program  logistics  consisted  of  78  flat 
plate  specimens  which  includes  one  standard,  specimen 
holding  fixtures,  ultrasonic  and  eddy  current  bolt  hole 
NDI  procedures,  specimen  scoring  sheets  and  technician 
evaluation  sheets. 

SLIDE  J-3 

The  screening  specimens  were  made  from  7075-T6  aluminum 
alloy  flat  plate  blanks  measuring  2"  wide  by  16"  long 
and  0.20"  thick.  Each  specimen  contained  ten  drilled 
and  countersunk  11/64-inch  diameter  holes.  Initially, 
smaller  diameter  pilot  holes  were  drilled  in  70  of  the 
blanks  which  would  contain  cracks.  Cracks  were  grown 
from  starter  notches  in  the  pilot  holes  using  a fatigue 
machine.  Of  the  770  holes  which  would  be  available  as 
inspection  sites,  123  of  them  would  contain  cracks.  The 
cracks  were  grown  in  70  of  the  samples  as  follows:  27 
samples  contain  only  one  crack,  33  samples  contain  two 
cracks  and  10  samples  contain  three  cracks.  The  cracks 
were  randomly  distributed  among  the  holes.  Seven  speci- 
mens contained  holes  without  cracks.  Cracks  were  grown 
in  lengths  providing  four  size  ranges  after  drilling 
the  holes  to  final  size.  The  four  crack  length  inter- 
vals were  0.050  to  0.100,  0.101  to  0.175,  0.176  to 
0.250,  and  0.251  to  0.350. 


Each  sample  was  serialized  on  one  end  with  an  identi- 
fication number.  Also  each  hole  in  the  sample  was 
identified  with  a digit  from  0 to  9.  Final  preparation 
included  painting  with  one  coat  of  epoxy  primer  plus 
one  coat  of  polyurethane  enamel,  then  reaming  the  holes 
to  0.191  inch  diameter  and  countersinking.  Following 
this,  the  specimens  were  inspected  with  ultrasonic, 
eddy  current  and  visual  techniques  to  assure  detect- 
ability of  the  cracks  with  the  NDI  techniques  provided, 
but  undetectable  with  the  unaided  eye. 


SLIDE  J-4 


Data  sheets  for  technician  reporting  of  inspection 
results  contain  scaled  drawings  of  each  specimen  with 
provisions  for  recording  the  specimen  identification. 
The  technician  simply  marked  the  hole  where  he  found  a 
crack  and  the  position  of  the  crack  on  the  chart. 

SLIDE  J-5 

A form  was  used  by  the  Lockheed  test  monitor  accom- 
panying the  program  for  grading  the  results  of  the 
technician's  performance  on  the  samples.  These  forms 
listed  each  crack  by  hole  and  specimen  number,  crack 
size  range,  and  additional  information  about  the 
inspector,  the  test  site  and  NDT  method.  Columns  were 
provided  for  recording  whether  the  technician  found  or 
missed  each  flaw. 
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SLIDE  J-6 


Relatively  easy,  straightforward  inspection  procedures 
were  provided  the  technician  with  step-by-step  instruc- 
tions for  performing  the  ultrasonic  shear  wave  and  eddy 
current  bolt  hole  techniques  on  the  samples.  Instruc- 
tions for  equipment  set-up  and  calibration  were 
provided  in  the  "Have  Cracks"  NDI  Manual. 

SLIDE  J-7 


The  manner  in  which  the  inspection  of  the  screening 
samples  was  conducted  was  similar  to  the  "Have  Cracks" 
inspections.  Only  technicians  who  were  assigned  to  the 
"Have  Cracks"  program  participated  in  the  screening 
program.  The  technician  was  given  the  samples,  the 
inspection  procedures,  calibration  and  standards  and 
the  reporting  sheets  and  he  proceeded  as  he  normally 
would. 

SLIDE  J-8 


Hie  screening  program 
"Have  Cracks"  program 
was  conducted  at  four 


was  first 
at  Tinker 
add i tional 


technicians  participated.  Sixteen 


incorporated  into  the 
AFB  and  subsequently 
bases.  A total  of  26 
inspection  tasks  were 
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completed  for  the  eddy  current  technique  and  9.25  tasks 
were  completed  for  the  ultrasonic  technique.  I might 
add  that  20  of  these  tasks  were  performed  at  two 
separate  depots  visited.  The  remaining  six  were  per- 
formed at  MAC  and  TAC  bases. 


SLIDE  J-9 

Tbe  results  were  tabulated  and  were  later  compared  to 

the  results  of  like  inspections  on  the  regular  "Have 

Cracks"  samples.  Actually,  the  results  of  all  techni- 
cians were  combined  on  a graphic  plot  in  which  perfor- 
mance on  the  screening  samples  using  each  NDI  technique 

was  plotted  against  performance  on  the  "Have  Cracks" 

samples.  One  technician  provided  one  data  point  on 
these  plots,  and  is  presented  in  terms  of  percentage  o£ 
flaws  found.  A regression  curve  of  approximately  45 
resulted  for  the  ultrasonic  inspection  - which  is  just 
about  ideal.  This  indicates  a strong  potential  for  us- 
ing screening  samples  to  measure  technician  proficiency 
in  ultrasonic  NDI.  On  the  other  hand,  the  technicians 
using  eddy  current  bolt  hole  techniques  detected  essen- 
tially 100  percent  of  the  cracks  in  the  screening  sam- 
ples, thus  failing  to  yield  a reasonable  correspondence 
between  inspections  on  these  samples  and  structure. 

SLIDE  J-10 

CONCLUSIONS  ■ As  can  be  seen  by  the  plots  for  the  eddy 
current  and  ultrasonic  techniques,  comparing  perform- 
ance on  the  screening  samples  to  performance  on  the 
structure  samples,  our  initial  hopes  appear  to  be  at 
least  partially  fulfilled.  That  is,  we  hoped  that  the 
experiment  would  show  us  that  flat  plate  samples  could 
be  used  to  predict  the  performance  a given  technician 
might  be  expected  to  yield  during  inspection  of  real 
structures.  This  appears  to  be  the  case  for  the  ultra- 
sonic technique.  On  the  other  hand,  the  eddy  current 
method  yielded  an  unsatisfactory  lack  of  correspon- 
dence, showing  that  the  technicians  performed  consider- 
ably better  on  the  screening  samples  than  on  the 
structure  samples.  Perhaps  there  are  some  good  reasons 
for  this.  For  example,  the  fastener  holes  in  the 
screening  samples  were  clean,  smooth  new  holes,  whereas 
the  holes  in  the  structure  samples  contain  various 


amounts  of  damage  due  to  fastener  installations  and 
removals,  particularly  for  the  interference  fit  holes. 
Hence,  noise  is  produced  in  the  eddy  current  display 
which  inhibits  detection  of  cracks  in  the  structure 
samples.  We  do  not  feel  that  the  question  is  settled, 
however,  for  either  NDI  method.  Greater  participation 
would  have  given  us  more  data  points,  resulting  in 
better  definition  of  the  comparison  curves.  We  feel 
that  there  is  high  potential  value  for  improving  our 
inspection  program  in  a screening  program  such  as  this 
and  we  need  to  pursue  it  further. 
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NDi  Proficiency  Screening  Program 
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OBJECTIVE: 

• TO  DETERMINE  IF  TECHNICIAN  PROFICIENCY  CAN  BE 
EASILY  MEASURED  AND  PREDICTED 


• TECHNICIAN  PERFORMANCE  ON  FIAT  PLATE  SAMPLES 
COMPARED  TO  PERFORMANCE  ON  COMPLEX  STRUCTURE 
SAMPLES 

FUNDED  BY  AFML,  IMPLEMENTED  IN  AFLC /LOCKHEED 
"HAVE  CRACKS  - WILL  TRAVEL"  PROGRAM 
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Program  Logistics 


1 y •• 


• 78  FLAT  PLATE  SAMPLES 

• SAMPLE  HOLDING  FIXTURE 

• NO  I PROCEDURES 

• SAMPLE  SCORE  SHEETS 

• TECHNICIAN  PERFORMANCE  SHEETS 
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7W5-TI  ALUMINUM 

NO.  SPECIMENS  I NO.  CRACKS /SPEC  I TOTAL  CRACKS 


18  CRACKS 


77  SPECIMENS 


Technician  Evaluation  Form 
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NDi  Procedures  for  Screening  Samples 


ANALYSIS  OF  PROGRAM  RESULTS 
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We  have  noted  previously  that  there  are  wide  variations 
in  the  flaw  detection  probabilities  measured  for  indi- 
vidual inspectors  at  a given  flaw  site.  Aside  from  the 
fact  that  different  flaws  may  vary  - because  of  their 
peculiar  character istics  - in  their  susceptibility  to 
detection,  we  certainly  were  interested  in  whether  our 
data  could  show  causes  for  this  variability.  Thus,  we 
looked  at  some  factors  that  should  affect  proficiency. 

We  had  collected  data  on  personal  characteristics  and 
experience  of  Air  Force  inspectors  which  should  be 
directly  relatable  to  their  NDI  proficiency.  We  will 
look  at  three  cases  that  indicate  how  NDI  performance 
relates  to  years  of  NDI  experience,  number  of  hours  of 
formal  NDI  training  and  the  number  of  inspections  con- 
ducted per  month  for  one  method.  We  looked  at  a compos- 
ite of  results  from  all  participating  technicians  at 
the  21  bases  and  plotted  these  in  terms  of  the  Total 
Percentage  of  Finds  versus  each  of  the  three  variables 
mentioned  above,  using  the  data  from  the  eddy  current 
surface  scan  NDI  method.  This  was  done  without  regard 
to  sample  type  or  flaw  size. 

The  first  plot  shows  the  Total  Percentage  of  (Flaw) 
Finds  versus  Years  (of  NDI)  Experience.  One  would 
expect  NDI  performance  to  improve  with  experience.  This 
plot  does  not  support  this  premise.  Maybe  there  is  a 
weak  trend  in  that  regard.  One  thing  it  seems  to  indi- 
cate is  that  the  really  capable  technicians  become  good 
rather  early,  but  many  of  these  may  not  become  career 
Air  Force  personnel  - leaving  the  Air  Force  after  3 or 
4 years  to  apply  their  skills  elsewhere.  One  other 
indication  is  that  some  of  those  who  do  become  career 
personnel  either  have  not  sharpened  their  skills  in  the 
eddy  current  method  or  are  really  not  suitable  individ- 
uals for  applying  it.  These  thoughts  are  presumptious 
and  are  not  to  be  considered  as  conclusive.  The  main 
thing  is  to  note  that  there  is  generally  no  predictable 
trend  evident  from  a comparison  based  on  years  of  NDI 
ex  per ience . 
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The  second  plot  shows  Total  Percentage  of  Finds  versus 
Hours  of  Formal  (NDI)  Training.  We  asked  each  partici- 
pating technician  to  list  the  number  of  hours  he  has 
spent  in  formal  NDI  training.  This  could  include  non- 
military training,  but  would  not  include  on-job- 
training. I'm  sure  there  are  some  inaccuracies  in  these 
figures  since  the  technicians  entertained  numerous 
concepts  of  what  constitutes  formal  NDI  training.  But 
assuming  that  the  figures  are  representative  of  the 
overall  situation,  the  hours  of  formal  NDI  training 
were  examined  for  its  influence  on  performance.  Even 
though  no  definite  trend  is  evident,  there  is  some 
indication  that  a weak  inverse  relationship  exists, 
i.e.,  those  with  more  formal  training  do  not  perform 
as  well  as  those  with  less.  At  any  rate,  the  plot 
clearly  indicates  that  formal  training  as  now  conducted 
and  in  excess  of  basic  familiarization  has  questionable 
value  in  improvement  of  the  techician's  skills.  This 
does  not  mean  that  additional  training  degrades  per- 
formance, but  rather  that  additional  formal  training  of 
the  types  reported  does  not  automatically  improve 
proficiency. 

SLIDE  K-3 

The  thiid  plot  shows  Total  Percentage  of  Finds  versus 
the  number  of  Eddy  Current  Inspections  per  month. 
Again,  there  is  no  clear  trend  evident  in  the  data. 
There  is  only  a weak  indication  that  per formance  im- 
proves with  more  intensive  practice.  However,  many 
technicians  are  able  to  perform  very  well  even  when 
they  do  not  have  the  opportunity  to  apply  the  ET  method 
frequently.  It  should  be  noted  that  the  eddy  current 
surface  scan  technique  is  relatively  simple  and  once 
the  skill  is  acquired  it  does  not  require  extensive 
practice  to  maintain  it.  However,  poor  performance 
(i.e.,  less  than  40%  of  finds)  seems  to  disappear  when 
the  technician  has  the  opportunity  to  apply  it  with 
moderate  frequency  (more  than  ten  times  per  month). 
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EODY  CURRENT  SURFACE  SCAN  PERFORMANCE 


VS  YEARS  EXPERIENCE 


IOTAI  * FINOS, 
ET  SURFACE 
PROBE.  21  BASES 


TUB  HOT  B A COMAOSITC 
WITHOUT  O&ATO  TO  SAMTtE 
TYFT  OUTLAW Jltt 


YEARS  EXPERIENCE 


EODY  CURRENT  SURFACE  SCAN  PERFORMANCE 
VS  HOURS  FORMAL  TRAINING 


TOTAL  » FINOS, 
ET  SURFACE 
PROBE.  21  BASES 


THIS  HOT  IS  A CO^EOSITE 
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FRACTURE  MECHANICS  REQUIREMENTS  VERSUS  EXISTING  NDI 

CAPABILITY 


SLIDE  L-l 

The  materials  and  designs  used  in  modern  aircraft  do 
not  tolerate  large,  crack-like  flaws.  We  can  no  longer 
wait  until  a crack  is  visually  obvious,  then  stop-drill 
it,  and  drive  on  as  we  once  did.  Critical,  highly 
stressed  components  in  some  of  the  newer  aircraft  will 
not  tolerate  even  a one-half  inch  long  crack  before  it 
would  be  unsafe  for  further  flight.  This,  of  course,  is 
why  we  now  depend  heavily  on  nondestructive  inspection 
to  insure  the  structural  integrity  of  our  hardware.  But 
our  inspection  requirements  are  becoming  more  demand- 
ing, we  are  looking  for  smaller  defects,  inspecting 
larger  areas,  (for  example:  inspection  of  16,000  fast- 
ener areas  on  the  C-5  wing),  and  insisting  on  greater 
confidence  in  our  inspection  results.  In  fact,  all  new 
aircraft  are  now  being  designed  to  specific  damage  tol- 
erance criteria  with  fracture  mechanics  technology. 
This  technology  specifies  the  precise  size  of  defect 
which  in-service  inspections  must  find  and  the  specific 
degree  of  reliability  and  confidence  with  which  it  must 
be  found.  Due  to  the  tremendous  costs  in  designing  and 
procuring  new  aircraft,  we  now  must  be  concerned  with 
the  durability  of  our  present  aircraft  over  extended 
service  lifetimes.  As  we  continue  to  demand  higher  and 
longer  performance  in  our  aircraft,  our  reliance  on  NDI 
to  insure  structural  integrity  will  increase.  From 
this,  I think  it  becomes  obvious  why  we  must  improve 
our  NDI  reliability  to  increase  our  confidence  that  we 
can  find  all  the  harmful  defects  in  both  new  and  aging 
hardware . 

SLIDE  L-2 

This  is  a typical  example  of  fracture  mechanics  techno- 
logy applied  to  our  present  day  fleet  of  aircraft. 
Shown  here  is  a crack  growth  curve  for  a specific 
location  on  the  C-5A  wing  where  chordwise  cracks  are 
predicted  to  originate  from  fastener  holes  in  the  wing 
panel  spanwise  splices.  Such  a typical  configuration  is 
shown  in  the  upper  right-hand  corner.  As  you  can  see 
from  the  predicted  growth  of  the  crack,  we  cannot  wait 
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until  it  reaches  a size  where  we  can  visually  detect 
it.  At  such  a point  in  time  the  crack  grows  too  fast, 
rapidly  becoming  unstable  and  the  structure  is  unsafe 
for  further  flight.  But  the  point  here  is:  where  can  we 
reliably  depend  on  NDI  to  detect  such  cracking?  If,  for 
example,  as  shown  by  the  lower  horizontal  dotted  line, 
we  can  reliably  expect  NDI  to  detect  a crack  an  eighth 
of  an  inch  in  length  in  this  location,  then  obviously 
we  should  begin  our  inspections  at  around  12,000  flight 
hours  and  perhaps  we  might  have  two  or  three  opportuni- 
ties during  normal  major  overhauls  to  detect  such  a 
crack  before  it  progresses  to  the  extent  where  it  ap- 
proaches instability.  But  how  comfortable  do  we  feel 
with  this  eighth  of  inch  crack  size  being  reliably  de- 
tected. As  you  have  seen  from  the  data  presented  earli- 
er today,  an  eighth  of  an  inch  crack  is  far  below  the 
average  crack  size  normally  detected  reliably  by  most 
routine  aircraft  NDI  inspections. 

SLIDE  L-3 


This  is  a current  inspection  requirement  on  the  T3B 
aircraft  currently  existing  in  the  Air  Force  inventory. 
Here,  in  a radius  area  at  the  wing  root,  the  require- 
ment exists  to  conduct  an  eddy  current  inspection  every 
100  flying  hours  for  the  detection  of  fatigue  cracks. 
The  fracture  mechanics  analysis  showed  a requirement 
for  detection  of  0.035  inch  cracks  in  this  particular 
area.  That  is,  a crack  of  0.035  of  an  inch  in  length 
will  grow  to  its  critical  size  within  two  inspection 
periods  if  left  undetected.  With  this  particular  in- 
spection requirement,  we  feel  very  confident  that  we 
can  realistically  detect  0.035  inch  cracks  with  a high 
degree  of  reliability  because  the  area  is  readily  ac- 
cessible, the  cracks  are  open  to  the  surface  and  we 
have  both  an  eddy  current  and  a penetrant  inspection 
method  available  for  use.  As  you  can  see  the  area  to  be 
inspected  is  very  isolated  and  relatively  small,  less 
than  2 inches  in  total  length.  All  of  this  we  feel  re- 
sults in  a very  direct  and  highly  reliable  inspection 
application.  This  is  considerably  different  than  in- 
specting hundreds  or  even  thousands  of  potential  crack 
sites  on  such  aircraft  as  the  KC-135  and  C-5  aircraft. 
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I would  like  to  bring  to  your  attention  some  of  the 
problems  we  have  encountered  in  applying  NDI  relia- 
bility data  to  meet  fracture  mechanic  requirements.  One 
such  problem  is  the  obvious  gap  between  the  flaw  size 
requiring  detection  and  the  flaw  size  which  can  be  de- 
tected with  a high  degree  of  reliability  and  confi- 
dence. Another  problem  exists  with  the  level  of  ciack 
detection  reliability  required,  that  is,  90%  probabili- 
ty of  detection  at  a confidence  level  of  95%. 

SLIDE  L-6 

Let  me  give  you  a brief  idea  of  exactly  what  90%  rel- 
iability at  a 95%  confidence  factor  actually  means.  If 
you  were  required  to  demonstrate  to  me  that  you  could 
find  a one-eighth  inch  defect  at  90  - 95  you  would  have 
to  find  that  defect  29  times  out  of  29  attempts.  You 
would  not  be  able  to  miss  that  defect  even  once  in  29 
tries.  If  you  did  miss  it  once  than  you  would  have  to 
find  it  45  times  out  of  46  attempts,  or  with  two  misses 
and  59  successes  in  61  trials,  and  so  forth.  The  per- 
centages work  out  to  be  considerably  higher  than  90% 
probability  of  detection  in  order  to  demonstrate  that 
extremely  high  confidence  bound  of  95%.  The  95%  confi- 
dence bound  is  a statistical  level  of  confidence  and  in 
reality  a 100%  confidence  level  can  never  be  reached  as 
it  is  an  asymptotic  limit  value.  Some  people  think  that 
a 50%  confidence  limit  means  a 50  - 50  chance  of  suc- 
cess. This  obviously  isn't  so,  as  can  be  seen  from  the 
lower  half  of  this  slide.  In  order  to  demonstrate  90% 
reliability  at  a 50%  confidence  level  that  one-eighth 
of  an  inch  flaw  must  be  found  seven  times  out  of  seven 
attempts.  If  it  is  missed  once  in  seven  tries  then  it 
must  be  found  sixteen  times  out  of  seventeen  attempts, 
and  so  on.  As  you  can  see,  even  a 50%  confidence  bound 
is  a relatively  high  level  of  confidence  statistically 
speaking  . 
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This  is  another  representation  of  the  same  information 
shown  in  the  previous  slide  and  displays  the  number  of 
successful  attempts  of  crack  detection  required  to 


demonstrate  90%  reliability  at  the  variously  indicated 
confidence  levels  of  50%,  70%,  90%,  95%  and  99%.  When 
you  consider  that  in  a demonstration  program  not  every 
sample  to  be  inspected  will  contain  a defect  requiring 
detection  and,  in  fact,  it  is  even  desirable  to  have  a 
much  larger  number  of  unflawed  samples  mixed  with  the 
flawed  samples,  then  you  can  see  the  magnitude  in  terms 
of  time  and  expense  involved  in  demonstrating  90  - 95% 
detection  criteria.  I don't  propose  here  to  offer  what 
is  a more  realistic  confidence  level  for  inspection 
results  to  attain;  but  I am,  however,  raising  the  issue 
of  whether  95%  confidence  is  truly  a realistic  goal  in 
a maintenance  inspection  environment. 
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As  you  can  see  from  this  chart,  in  order  to  prove  or 
demonstrate  a minimum  of  90%  detection  capability  at 
any  statistical  confidence  level,  the  actual  demonstra- 
ted detection  rate  always  has  to  be  somewhat  higher 
than  90%.  We  have  to  be  very  careful  in  applying  any 
confidence  levels  at  all  to  our  inspection  results  and 
when  we  do  we  must  make  sure  that  we  know  precisely 
what  we  are  doing. 


Eddy  Current  Inspection  to  Detect 
0.035  Inch  Crack  Every  100  Hours 


Problem 


FM/NDI  Interface 


e Allowable  vs  Detectable 
Flaw  Size 


e Statistical  Level  90-95 
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FAMILIARIZATION  WITH  AVAILABLE  COMPUTER 
SLIDE  N-l 

The  purpose  of  this  talk  is  to  discuss  the  data  manage- 
ment system  and  the  analysis  procedures  that  were  dev- 
eloped for  processing  the  raw  inspection  data  acquired 
from  field  and  depot  bases  during  the  program.  These 
procedures  permit  a quantitative  determination  of  the 
significance  of  various  factors  affecting  Air  Force  NDI 
capability.  Tthe  management  system  and  the  procedures 
are  available  for  use  during  this  workshop  and  I hope 
that  all  of  you  can  take  advantage  of  this  capability 
while  you  are  here. 

The  effort  to  develop  the  data  management  system  and 
analysis  procedures  was  conducted  under  a separate  pro- 
gram from  the  NDI  reliability  program  and  was  funded  by 
AFML.  The  scope  of  the  effort  was  to  develop  computer 
procedures  for  the  storage,  retrieval  and  analysis  of 
program  data.  The  program  involved  the  design  of  a data 
base  system,  development  of  pre-processing  and  post- 
processing computer  programs,  implementation  of  the 
data  management  system  and  an  operational  checkout  of 
it,  an  initial  data  analysis  and  then  a transfer  of  the 
operational  programs  to  AFML  for  further  use. 

All  the  significant  raw  data  accumulated  from  the  field 
and  depot  bases  are  stored  under  specific  organized 
listings  in  ASD's  ITEL  AS/5  computer.  These  data  sets 
are  given  in  both  narrative  and  numerical  notation,  but 
are  not  necessarily  identified  in  the  computer  as  given 
on  this  slide.  The  listings  on  the  slide  show  the  cate- 
gories of  data  that  are  available.  They  include  the 
structure  samples,  NDI  methods,  crack  information,  base 
identification,  technician's  personal  and  background 
data,  local  weather  and  environment  data,  inspection 
dates  and  results,  and  listings  of  technicians  in  ac- 
cordance with  their  relative  performance  level. 

In  addition  to  the  raw  data  listings,  there  are  identi- 
fier words  by  which  the  operator  can  recall  specific 
listings  for  data  categories,  probability  of  detection 
plots  for  various  combinations  of  parameters  which  one 
may  desire  to  analyze. 
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Use  of  the  data  management  and  analysis  system  is  rela- 
tively simple.  Raw  data  in  both  narrative  and  numerical 
form  is  organized  by  the  system  and  allows  for  access 
by  an  Eng  1 ish- 1 i ke  syntax.  This  approach  allows  treat- 
ment of  data  without  special  programming  skills.  Output 
at  the  remote  terminals  can  be  selected  for  a graphic 
plot  or  a printed  format.  Computer  processed  informa- 
tion can  be  output  in  the  form  of  raw  data  listings 
which  are  selected  by  identifier  words,  probability  of 
flaw  detection  plots  also  selected  by  identifier  words 
and  variance  ratios  which  are  statistically  calculated 
for  parameter  combinations. 

SLIDE  N-3 

Let  me  say  a few  things  about  the  data  processing  sub- 
routines. The  variance  ratio  calculations  allow  the 
operator  to  determine  the  significance  of  changes  in 
flaw  detection  capabilities.  The  variance  ratios  com- 
pare changes  attributed  to  some  given  NDI  parameter  to 
influences  which  can  be  expected  to  occur  from  random 
data  scatter.  The  calculations  will  show  whether  the 
change  is  in  the  scatter  band  and  is  therefore  insigni- 
ficant or  whether  the  change  is  beyond  the  scatter 
band,  indicating  a significant  influence  of  the  para- 
meter. The  statistical  method  used  will  accommodate 
missing  values  in  the  data  matrix.  In  processing  var- 
iance ratio  calculations,  we  are  able  to  select  four 
parameters  at  once.  For  example;  what  would  be  the 
significance  on  flaw  detection  probability  of  using  the 
ultrasonic  technique  on  Fridays  at  the  depots  for  all 
program  flaw  sizes?  When  the  results  of  this  analysis 
are  compared  to  the  same  type  of  analysis  determined 
for  Wednesdays,  we  can  evaluate  the  possibility  that 
Friday  may  not  be  as  favorable  a day  as  Wednesday  for 
performing  ultrasonic  inspections  at  the  depots. 

We  can  produce  graphic  plots  of  flaw  detection  proba- 
bility versus  a flaw  characteristic,  such  as  length, 
flaw  area  or  aspect  ratio.  These  plots  can  be  made  for 
selected  parameters  such  as  NDI  technique,  field  or 
depot,  structure  type,  technician  experience,  and  so 
on . 
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The  graphic  plots  are  statistical  representations  of 
the  data  for  flaw  detection  probability  versus  selected 
parameters.  Curves  are  generated  for  mean  values  and 
one-side  lower  confidence  bounds  at  95%,  90%  and  50% 
levels  of  significance.  Any  parameter  in  the  data  list 
can  be  selected  to  examine  its  apparent  relationship  to 
the  probability  of  detection  and  compared  to  chance 
variation  with  the  " F"  test  for  significance. 

SLIDE  N-4 

The  post-processing  program  can  be  used  to  investigate 
interrelationships  for  selected  groups  of  data  to  de- 
termine their  effects  on  NDI  reliability.  These  exam- 
ples indicate  parameters  for  which  graphic  plots  of 
probability  of  detection  versus  flaw  size  can  be  gen- 
erated and  examined  for  significance. 

Ttie  probability  of  detection  versus  flaw  size  can  be 
plotted  for  each  NDI  method  and  each  structure  type  and 
these  plots  can  be  made  using  the  combined  results  of 
all  technicians  or  using  only  the  results  from  techni- 
cians in  the  top  50,  25,  10  or  5 percent  performance 
levels. 

Then  we  can  examine  the  effect  that  various  personal 
factors  characteristic  of  the  technician  may  have  on 
NDI  performance.  POD  curves  can  be  plotted  for  para- 
meters selected  from  the  technicians'  experience,  skill 
level,  education,  NDI  training  and  experience,  his  age, 
marital  status,  physical  type  and  so  on. 

We  can  make  plots  for  the  field  base  and  the  depots, 
then  make  comparisons  for  significance  between  these 
two  types  of  facilities.  We  can  also  look  at  individual 
bases  and,  grouping  the  bases  according  to  type  of 
command,  look  at  each  command.  We  can  compare  perfor- 
mance between  days  of  the  week,  to  determine  if  there 
is  a significant  improvement  or  deterioration  in  per- 
formance for  given  days  of  the  week.  In  instances  where 
weather  conditions  are  considered  a possible  factor,  we 
can  determine  whether  differences  are  influenced  by 
extreme  weather  variables. 
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Now,  it  is  one.  thing  to  determine  that  significant 
differences  exist  for  some  parameters  when  compared  to 
other  parameters;  but  it  is  quite  another  thing  to 
determine  the  basis  for  the  difference.  For  example,  if 
we  find  there  are  significant  differences  in  perform- 
ance between  two  particular  commands  or  between  bases 
within  any  command,  what  do  we  attribute  these 
differences  to?  Perhaps  we  can  then  compare  differences 
in  training,  overall  experience,  operational  aspects, 
or  some  other  potential  factor  to  evaluate  the  reason 
one  base  exhibits  better  performance  than  an  otherwise 
similar  base. 


Let  me  point  out  that  once  we  have  analyzed  the  data 
performance  levels  and  significant  differences  our 
concern  now  is  with  determining  the  reasons  or  causes 
for  good  or  bad  performance  and  the  differences  between 
performance  levels.  Only  by  finding  these  basic  causes 
can  we  devise  effective  and  efficient  efforts  to  im- 
prove the  NDI  capability.  Certainly,  we  would  like  for 
the  capability  to  be  as  high  as  reasonably  possible  and 
we  would  like  for  the  performance  level  to  be  uniformly 
high  across  all  Air  Force  bases  and  Commands.  Our  ques- 
tion is  "What  things  must  we  do  to  bring  this  about?" 
What  changes  must  we  make  in  technician  selection,  for- 
mal NDI  training,  job  assignment,  on- job-training , and 
so  on,  to  make  worthwhile  and  permanent  improvements? 


Our  computerization  of  data,  storage,  retrieval  and 
processing  has  been  designed  to  allow  us  to  make 
statistical  analyses  for  a broad  range  of  conditions. 
These  analyses  will  tell  us  what  the  average  perform- 
ance is  for  the  selected  set  of  conditions  and  can  tell 
us  where  significant  variations  exist.  But  it  is  our 
(insight  and  ingenuity  that  will  allow  proper  use  to  be 
made  of  the  analyses  to  find  the  causes,  make  correct- 
ional changes,  and  thereby  achieve  the  true  objectives 
of  this  effort. 
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Computer  Utilization 
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Data  Processing  Routines 

• VARIANCE  RATIO  CALCULATIONS  FOR  UP  TO  FOUR 
SIMULTANEOUS  PARAMETERS 
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Graphic  Plots  of  POD  vs  Flaw  Size 
for  Selected  Parameters 
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Section  IV 

WORKSHOP  TASK  GROUPS 

Task  Group  Function 

The  heart  of  the  Workshop  was  the  Task  Groups  composed 
of  the  attendees.  The  Task  Groups  were  established  to 
address  specific  aspects  of  NDI  which  were  of  general 
interest  to  both  industry  and  the  Air  Force.  The  object 
of  these  groups  was  to  consider  the  results  of  recent 
programs  designed  to  measure  and  evaluate  the  relia- 
bility of  NDI  and  to  determine  what  can  be  done  to 
improve  NDI  effectiveness.  Leaders  for  each  of  the  Task 
Groups  were  preselected  from  the  preregistered  attend- 
ees by  the  Workshop  organizers  based  on  their  particu- 
lar background  and  interest  in  the  specific  area  con- 
cerned . 

Preparation  and  distribution  of  Task  Group  Packages  was 
accomplished  to  initiate  in-depth  studies  of  the 
USAF/Loc kheed  NDI  Reliability  Program.  Additionally, 
quantitative  results  to  questions  and  problem  areas 
were  provided  by  computer  access  to  the  Reliability 
Program  data  bank  through  a remote  terminal  located  in 
the  Workshop.  Curve  plotting  and  data  printouts  speci- 
fied by  selected  variables  were  made  available  on 
request  from  data  processing  subroutines  in  the  com- 
puter. The  Workshop  goal  was  to  formulate  concensus 
recommendations  to  further  the  improvement  of  NDI  and 
to  suggest  programs  which  would  provide  a more  quanti- 
tative evaluation  of  NDI  and  to  implement  improvements 
in  maintenance  inspection  operations.  A detailed 
explanation  of  each  of  the  eight  areas  covered  by  the 
Task  Groups  is  given  in  the  following  section. 

Task  Group  Breakdown 

Task  Group  I - Personnel 

This  task  group  was  concerned  with  the  problems  and 
methods  of  improving  the  initial  selection  of  inspec- 
tion personnel  and  the  maiiuenance  of  high  levels  of 
capability  of  existing  personnel.  Basic  concerns  in- 
cluded discussions  of  personnel  background  and  char- 
acteristics which  may  be  indicators  of  good  NDI 
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technicians.  The  task  group  considered  these  possi- 
bilities as  well  as  questions  such  as  how  may  NDI 
personnel  be  motivated  to  always  do  their  best;  what 
incentives  would  induce  technicians  to  improve  their 
capabilities,  and  what  human  factors  affect  NDI  opera- 
tions and  how  can  they  be  controlled?  Task  group 
participants  were  given  the  opportunity  to  discuss  and 
answer  these  questions  based  on  their  past  experience 
and  expertise. 

Task  Group  II  - Training 

This  task  group  explored  both  formal  and  on-the-job 
training  of  NDI  personnel,  emphasizing  the  factors  that 
would  produce  better  qualified  personnel  and  promote 
reliable  inspections.  The  quality,  content  and  emphasis 
on  initial  and  continuing  formal  training  and  on-the- 
job  training  was  addressed.  Other  items  discussed  in- 
cluded: Methods  to  obtain  an  optimum  balance  between 
NDI  theory  and  practice  in  the  training  of  both 
laboratory  and  field  personnel;  should  specialized  or 
generalized  training  be  pursued?;  methods  for  measure- 
ment of  NDI  training  effectiveness;  evaluate  means  to 
minimize  cost  of  NDI  training;  determine  how  much 
practical  training  is  necessary. 

Task  Group  III  - Operations 

This  task  group  addressed  NDI  operational  factors  that 
can  lead  to  improved  NDI  reliability.  Topics  of  discus- 
sion involved  the  optimization  of  the  NDI  task  size, 
job  scheduling  at  the  most  favorable  time  of  day  or 
week,  desirability  of  working  technicians  in  pairs  and 
definition  of  key  environmental  factors  for  greater 
efficiency.  Other  areas  explored  included  the  recogni- 
tion of  NDI  as  a discipline  within  the  organizational 
structure;  the  relationship  of  the  depot  to  field  units 
in  terms  of  schedule  time  for  each  NDI  task,  in  supply 
of  standards  and  equipment  and  in  the  validation  of 
techniques  or  T.O.'s  for  field  use.  This  task  group 
also  addressed  the  problem  of  personnel  training, 
whether  personnel  were  assigned  primary  or  secondary 
duty  as  NDI  technician  and  the  need  for  qualification 
in  all  methods  of  NDI. 
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This  task  group  was  concerned  with  the  certification 
and  qualification  of  NDI  personnel.  Questions  consider- 
ed by  this  group  included:  Would  certification  improve 
NDI  reliability?;  should  laboratory  as  well  as  field  or 
shop  NDI  personnel  be  certified  and  how  should  qualifi- 
cation be  attained  and  how  measured?  How  often  should 
certification  be  required  and  who  should  conduct  the 
verification  functions?  Since  certification  demon- 
strates knowledge  of  the  principles  and  practices  of 
NDI,  should  the  technician  also  be  required  to 
demonstrate  his  skill  in  performing  practical  NDI? 

Task  Group  V - Equipment 

This  task  group  was  concerned  with  potential  improve- 
ments in  NDI  equipment  designs  for  greater  compati- 
bility with  human  factors  in  search,  control  and 
display.  Other  considerations  included  the  reduction  of 
operator  dependence  in  flaw  detection  systems,  perman- 
ently recording  inspection  data  for  improved  detection 
and  interpretation,  equipment  design  considerations  for 
defect  search  and  characterization  modes,  design  and 
availability  of  inspection  standards,  and  the  effect  of 
equipment  performance  on  total  NDI  reliability.  Is  it 
practical  to  design  equipment  for  compatibility  with 
human  capabilities  rather  than  trying  to  adapt  humans 
to  fixed  equipment  designs? 

Task  Group  VI  - Reliability  Measurement/Modeling 

This  task  group  explored  the  possibility  of  translating 
the  data  and  results  from  the  "Have  Cracks"  Program  and 
similar  efforts  to  other  NDI  applications.  Could  a 
model  be  developed  to  predict  the  relaibility  of  a 
specific  nondestructive  inspection?  Determine  ap- 
proaches to  modeling  which  would  improve  measurement 
programs  such  as  "Have  Cracks"  Program  or  extend  it  to 
broader  representative  structure.  Based  on  data  obtain- 
ed in  "Have  Cracks"  Program,  could  a simple  NDI  test 
such  as  the  screening  samples  be  developed  to  effec- 
tively replace  more  extensive  evaluation  programs? 
Should  the  inspectors  know  they  are  participating  in  an 
evaluation  program?  Define  parameters  considered 
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essential  to  an  objective  modeling  approach  to  NDI 
reliability  measurement. 


Task  Group  VII  - Fracture  Meehan i cs/NDI  Interrela - 

tionship 

This  task  group  reviewed  the  relationship  and  use  of 
NDI  reliability  data  in  structural  design  based  on 
fracture  mechanics  design  principles.  Discussions 
involved  the  interrelationship  of  fracture  mechanics 
and  NDI  with  regard  to  transfer  of  informmation  and 
communication,  the  limitations  and  capabilities  of  each 
discipline,  and  the  effect  of  the  reliability  program 
on  structural  design  philosophy.  The  existing  NDI 
reliability  data  were  studied  with  regard  to  current 
FM/NDI  design  practices  which  assume  a minimum  detect- 
able flaw  size.  The  need  for  95%  confidence  level  on 
NDI  reliability  was  reviewed  and  the  requirements  for 
both  confidence  level  and  detection  of  minimum  flaw 
size  were  assessed  for  realistic  application  in 
aircraft  safety.  Further  discussions  involved  the 
necessity  for  timely  dialogue/communication  between 
NDI,  fracture  mechanics  and  design  organizations  to 
ensure  compatibility  between  requiremens  and  practical 
capability. 

Task  Group  VIII  - Data  Analysis 

This  task  group  was  concerned  with  the  analysis,  inter- 
relationships, presentation  and  application  of  existing 
NDI  reliability  data.  This  included  problems  such  as 
dealing  with  false  calls,  the  construction  and  use  of 
detection  probability  curves,  analyses  of  variance  or 
co-variance  and  determination  of  confidence  limits,  and 
how  to  most  effectively  format  data  relationships. 
Discussions  also  centered  or  the  best  way  to  determine 
the  significance  of  variations  in  NDI  reliability  data 
using  the  co-variance  technique  and  alternate  methods. 
Parametric  studies  were  discussed  relative  to  present- 
ing NDI  reliability  program  results  in  various  formats 
compatible  with  the  desired  use  of  the  data. 
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Section  v 

TRANSCRIPTION  OF  TASK  GROUP  REPORTS 


Business  Meeting,  Government/Industry  Workshop  on  NDI 
Reliability,  held  at  the  Shamrock  Hilton  Hotel, 
Houston,  Texas,  on  August  4,  1978,  beginning  at  8:30 

a.m.,  reported  in  shorthand  by  Kathleen  Alford,  Regis- 
tered Professional  Reporter,  Notary  Public,  Harris 
County,  Texas. 

I.  PERSONNEL 

TASK  GROUP  LEADERS: 


R.  L.  Wolford 

S.  I.  Shelton 
Bob  Erwin 


R.  L.  Wolford,  Summary  for  the  Personnel  Group:  The 
task  for  which  Sam  Shelton  and  Bob  Erwin  and  I were 
assigned  was  identified  as  Personnel.  The  charter  for 
that  group  was  to  determine  what  type  of  an  individual 
might  best  become  a satisfactory  NDI  technician.  We  had 
some  very  interesting  discussions,  which  I am  sure  all 
of  the  groups  must  have  had.  There  was  considerable 
amount  of  interest,  considerable  amount  of  varied 
discussions  on  what  the  individual  should  have.  Every- 
thing from  motivation  --  and  there  were  those  who 
thought  that  the  only  motivation  was  money.  There  were 
those  who  had  the  feeling  that  there  were  a lot  of 
other  things  much  more  deeply  of  concern  than  money  — 
that  money  became  a secondary  thing  if  you  were  suc- 
cessful in  the  other  items,  the  money  would  be  there. 
We  were  fortunate  in  having  in  the  group  representa- 
tives from  the  Canadian  Air  Forces,  Royal  Air  Force,  as 
well  as  our  own  Navy  and  Air  Force  personnel.  We  were 
able  to  make  comparisons  between  the  methods  utilized 
in  those  military  services  in  the  selection  of  person- 
nel for  NDI  work.  We  had,  of  course,  representation 
from  industry.  There  is  a vast  latitude  of  difference 
between  all  of  those  organizations  that  I mentioned.  We 
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have  put  together  a couple  of  viewgraphs.  This  view- 
graph  identifies  in  an  abbreviated  form  what  we  believe 
our  charter  was:  To  determine  the  methods  required  to 
improve  the  initial  selection  of  NDI  personnel  and  the 
maintenance  of  high  levels  of  capability  of  existing 
personnel.  How  can  NDI  personnel  be  motivated  to  do 
their  best? 

That  briefly  was  what  we  believed  our  charter  to  be. 
The  next  viewgraph  — these  were  the  major  items  that 
seemed  to  predominate  our  group.  Develop  a human 
factors  profile  which  identifies  basic  requirements. 
The  individual  is  to  have  integrity,  a sense  of  re- 
sponsibility, he  is  to  be  mature  enough  to  recognize 
his  responsibilities,  a pride  in  accomplishments.  He 
had  to  be  a person  who  was  self-motivated,  self- 
disciplined,  who  could  discipline  himself  to  work  in 
the  various  environments.  He  had  to  be  an  achiever.  He 
had  to  have  moral  convictions.  He  had  to  have  alle- 
giance and  decision-making  ability. 

Admittedly,  those  are  some  very  difficult  requirements 
to  find  in  a single  individual.  But  these  are  the 
things  in  summary  we  found  the  individual  profile  would 
require.  Integrity,  he  had  to  be  able  to  understand  his 
responsibility  and  to  be  moved  to  follow  those  convic- 
tions and  to  make  the  decision  that  is  necessary.  I 
think  basically  that  most  of  those  are  understood  by 
all  of  us  and  recognized  for  their  individual  responsi- 
bilities in  a person  performing  this  kind  of  a task. 


Then  there  are  certain  technical  requirements  that  the 
individual  should  have.  He  should  have  the  ability  to 
use  the  tools  of  the  profession.  He  should  be  able  to 
demonstrate  a proficiency  in  the  basic  sciences.  He 
should  have  training  or  work  experience  in  the  type 
material  or  structure  to  be  worked  with  prior  to  enter- 
ing the  NDI  field.  This  says  that  there  should  be  a 
training  period  in  which  the  individual  gets  extensive 
— and  I am  not  going  to  attempt  to  identify  what 
extensive  might  really  be  at  this  point.  He  should  have 
enough  experience  that  he  feels  free  in  the  use  of  the 
equipment.  He  understands  the  equipment.  He  under- 
stands the  purpose  that  the  particular  equipment  and 
technologies  have  and  what  their  limitations  are.  He 
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should  be  able  to  demonstrate  those  proficiencies  and 
all  of  the  disciplines  and  with  all  of  the  types  of 
equiment  that  he  is  going  to  be  expected  to  work  with 
and  the  type  of  structures  that  he  is  going  to  work  on 
— if  he  is  going  to  work  in  finished  products  such  as 
an  aircraft  and  that  type  of  thing.  Going  to  the  last 
item,  he  should  also  understand  something  about  air- 
craft structures.  There  should  be  some  education  in 
that  area.  Make  him  realize  the  importance  of  his 
inspections.  Where  critical  structure  areas  are  that  he 
has  a high  degree  of  responsibility  in  the  decision 
that  he  makes;  he  should  be  made  to  understand  that  he 
has  a responsibility,  a real  responsibility  for  that 
decision.  If  he  is  going  to  work  outside,  if  he  is 
going  to  work  in  a plant  on  more  of  a production  line 
NDI  job  where  he  is  doing  penetrant  or  mag  inspection 
as  opposed  to  maybe  doing  ultra-sonic,  eddy  current,  or 
radiography.  He  is  looking  at  individual  parts.  There 
is  a difference  between  that  type  of  inspection  and  one 
out  on  the  line  in  which  you  are  working  in  the  envir- 
onment that  are  quite  different  from  those  in  shop 
env  i ronment . 

It  makes  no  difference  if  he  is  going  to  work  on  pipe- 
lines or  building  construction  or  whatever.  He  should 
have  a good  knowledge  of  the  types  of  structures  and 
material  that  he  is  working  with  and  how  his  equipment 
is  going  to  perform  with  those  materials  in  those  types 
of  environments. 

Recommendations  : 

To  Develop  a Method  for  Evaluating  NDI  Personnel;  Re: 
The  Profile.  The  things  that  we  showed  you,  a means  of 
identifying  how  many  of  the  things  we  put  in  the  pro- 
file that  that  individual  has.  And,  if  necessary,  those 
could  be  put  in  a priority,  which  ones  of  them  you 
thought  were  the  most  important  and  where  you  wanted  to 
place  your  most  severe  measurement  of  the  individual. 

Initiate  personnel  motivation  criteria  which  will  pro- 
mote performance  excellence. 

Develop  recognition  for  outstanding  accomplishments. 
Some  of  the  suggestions  that  came  out  of  the  three 
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different  sessions  were  commendations  of  some  type, 
advancement  potential.  He  should  understand  what  his 
potentials  are.  How  far  can  I go  in  this  business? 
Management  acknowledgement,  I think  most  all  of  us 
enjoy  occasionally  to  be  recognized  for  having  accom- 
plished something  that  is  over  and  beyond  the  normal 
call  of  your  job  duty.  Somebody  at  least  was  aware  that 
you  did  a good  job.  You  did  it  ahead  of  time.  You  saved 
some  parts  that  might  have  been  scrapped  if  less  than 
more  critical  review  of  your  inspection  had  been  made. 

Develop  a feedback  system.  One  of  the  things  that  was 
brought  out  was  that  maybe  it  would  be  interesting  if 
we  took  those  suggested  questions  that  were  in  your 
packet  which  were  really  there  to  help  motivate  your 
discussions  in  your  respective  task  groups,  if  you  took 
those  same  questions  and  went  out  to  your  NDI  person- 
nel, whether  in  your  plant  or  out  on  an  air  base,  a 
station,  or  whatever,  and  had  your  people  look  at  those 
eight  tasks  and  the  kind  of  questions  that  were  there 
and  see  what  they  thought  their  job  task  was,  what  they 
found  and  would  suggest  to  you  as  being  desirable 
factors  that  would  help  them  to  do  a better  job,  where 
they  felt  that  maybe  they  were  short  changed  in  their 
training  program,  lack  of  proper  maintenance  of  their 
equipment.  These  are  some  suggestions  that  we  would 
identify  with  these  recommendations. 

This  is  a capsulated  presentation  of  the  three  sessions 

and  a very  brief  summary  of  what  we  felt  were  the  major 
items  to  come  out  of  that  particular  task. 

UNIDENTIFIED  SPEAKER: 

I would  like  tc  know  what  data  you  used  to  come  up  with 
these  items,  or  is  it  a collection  of  personal  opin- 
ions? 

MR.  WOLFORD: 


These  were  out  of  the  three  groups.  We  started  out  with 
the  first  question  in  the  first  task  group,  personnel. 
We  found  that  all  of  the  other  questions  that  they  had 
there  became  redundant  to  that  which  was  asked  in  the 
first  question.  These  were  the  things  that  either  were 


agreed  upon  by  the  groups  or  that  they  were  predominant 
in  the  suggestions  that  were  made  by  the  representa- 
tives in  those  three  sessions. 

UNIDENTIFIED  SPEAKER: 

Was  any  attempt  made  by  any  of  the  groups  to  prioritize 
even  in  the  way  of  grouping  these  various  factors  that 
have  been  identified  as  being  desirable? 

MR.  WOLFORD: 

The  question  was:  Was  there  any  attempt  to  prioritize 
the  suggestions  that  were  established  here,  whether 
they  be  I suppose  you  mean  in  the  profile. 

UNIDENTIFIED  SPEAKER: 

Yes,  that  list  of  desirables.  Which  are  the  most 
important? 

MR.  WOLFORD: 

I don't  think  that  I can  at  this  time,  Tom,  say  that 
there  was  as  a summary  of  the  group  to  have  priori- 
tized. I would  say  that  these  are  items  that  came  out 
that  were  predominant  in  the  discussions.  And  I think  a 
review  of  these  items  with  a group  of  people  that  are 
really  concerned  about  this,  to  study  this  may  be  in  a 
human  factors  type  thing  to  determine  which  of  these 
has  the  greatest  bearing  on  maybe  another  one, a sub- 
sequent one  and  so  on.  But  we  did  not  have  any  priority 
establ shed . 

The  question  was:  Where  there  were  general  statements 
made  on  the  view  graphs,  would  there  be  more  specific 
things  such  as  assuring  that  the  individual  knew  how  to 
read  blueprints. 

Yes,  because  we  were  not  sure  how  many  viewgraphs  or 
how  far  to  go  in  trying  to  break  down  at  this  particu- 
lar time  all  of  the  things  that  you  can  believe  were 
akin  to  this.  When  we  said  to  be  familiar  with  the 
structures,  certainly  we  felt  that  --  and  I say  the 
three  of  us  that  were  task  leaders  in  summarizing  this. 
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and  I suppose  with  some  degree  of  personal  opinion 
also,  these  questions  came  up.  You  introduce  your  own 
personal  feelings  also  --  that  the  individuals  had  to 
know  how  to  read  blueprints.  They  had  to  understand 
part  of  this  was  training  and  understanding  structure 
and  the  meaning  of  stucture,  the  importance  of  one  part 
of  a structure  for  instance  in  an  airframe  over  another 
part  of  it. 

What  is  fracture  critical,  for  example,  as  opposed  to  a 
maintenance  critical  and  so  on  down  the  line. 

The  individual's  question  was  that  there  needed  to  be 
some  definition  of  the  latitude,  I would  say,  of  what 
the  individual  could  deviate  from  the  technique  in  his 
i nspection . 

Now,  this  was  part  of  the  discussion.  How  far  do  you 
go?  Does  the  individual  have  the  responsi bi 1 i ty  to  make 
a final  decision?  Work  techniques,  was  he  working  from 
techniques  such  as  -36  or  other  specific  techniques 
developed?  Should  the  individual  have  any  prerogatives 
in  deviating? 

There  is,  I would  say,  a wide  latitude  in  some  areas 
whether  the  individual  should  have  those  prerogatives, 
that  there  is  a continuity  between  one  inspector  and 
another  inspector  on  a day-to-day  basis,  whether  they 
have  how  much  prerogative  to  deviate,  or  whether  you 
give  them  specific  guidelines  for  a particular  struc- 
ture to  assure  that  there  is  a continuity  in  the 
inspection  . 

Are  there  any  other  questions? 

UNIDENTIFIED  SPEAKER: 

I see  nothing  here  in  terms  of  physical  characteristics 
of  personnel.  Was  that  not  considered,  or  was  it  con- 
sidered adequately  covered  already? 

MR.  WOLFORD: 

These  were  discussed  briefly  as  being  part  of  the 
requirement.  They  were  not  identified  here  because  of 
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the  restriction  of  time  and  space  on  viewgraphs.  But, 
yes,  this  would  be  part  of  the  requirement.  There  would 
be  certain  physical  requirements  that  the  individual 
would  have  to  have.  There  are  certain  tasks  where  a 
person  with  certain  types  of  handicaps  would  function 
very  well.  Physical,  lost  arms  and  legs,  etc.,  some 
cases  have  to  sit  down  depending  upon  what  kind  of 
environment  the  individual  is  working  in.  Certainly 
working  out  on  a flight  line,  there  would  be  some 
restrictions  and  some  limits  to  be  placed  as  opposed  to 
working  in  a plant  where  a person  could  sit  at  a bench 
maybe  and  do  certain  types  of  ultrasonic,  conductivity, 
mag  particle  inspection  and  things  of  that  nature.  But 
we  did  not  get  into  any  detail  about  physical  require- 
ments specifically. 

UNIDENTIFIED  SPEAKER: 

In  the  area  of  technical  requirements,  training  and 
work  experience,  related  fields,  related  materials, 
prior  to  entering  the  NDI  field,  I think  it  is  a very 
important  point.  I was  wondering  what  specific  experi- 
ence was  discussed  and  deemed  to  be  important  to  move 
into  the  NDI  as  a machinist,  mechanic,  a maintenance 
man  on  the  line.  What  really  contributes  to  good  ex- 
perience and  background  of  this? 

MR.  WOLFORD: 

The  question  was:  In  the  technical  experience  prior  to 
going  into  the  NDI  field  was  there  any  discussion  as  to 
whether  he  should  have  machinist  experience,  to  be  a 
mechanic,  and  so  on. 

Yes,  there  was.  In  fact,  in  our  discussions  with  repre- 
sentation from  the  RAF  and  the  Canadian  Air  Force  in 
particular  we  spent  quite  a bit  of  time  in  the  session 
in  which  they  were  representatives  reviewing  the  tech- 
niques used  for  qualification  there.  And  for  the 
benefit  of  those  of  you  who  may  not  have  had  the 
opportunity  to  get  that  kind  of  information,  they  have 
a requirement  that  the  individual  has  to  have  been  in 
the  service  and  been  either  in  aircraft  structures  or 
an  engine  mechanic  for  six  or  seven  to  nine  or  ten 
years.  Then  he  can  apply  for  NDI.  He  then  goes  through 
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an  interview,  an  oral  interview,  which  is  conducted  by 
an  individual  but  that  individual  also  pulls  in  certain 
individuals  with  on-the-line  current  capability  in  the 
NDI  field  who  also  interrogate  the  individual.  If  he 
passes  that  interview,  he  is  then  put  on  a list,  a wait 
list.  And  when  the  opportunity  opens  up  for  another 
slot,  he  then  is  sent  to  school.  He  attends  school  for 
a prescribed  period  of  time  with  a prescribed  study  in 
all  of  the  disciplines.  He  may  be  weeded  out  in  that 
school  period.  After  he  graduates  from  the  school,  he 
then  has  a six-month  work  period  working  with  other  NDI 
people.  He  has  no  authority  to  sign  off  on  jobs  during 
that  six-month  period.  And  working  with  his  peers 
during  that  period  he  may  get  washed  out  at  that  time. 
If  he  demonstrates  certain  qualities  that  are  felt  not 
to  be  desirable,  he  could  be  washed  out.  After  the 
six-month  period  if  he  has  passed  all  of  that,  he  then 
has  the  authority,  and  there  they  have  the  total 
authority.  They  can  ground  an  airplane.  And  it  takes  a 
considerable  amount  of  doing  to  change  his  decision. 
This  places  a tremendous  amount  of  responsibility  on 
the  individual,  and  he  recognizes  when  he  goes  into  the 
program  the  extent  of  that  responsibility.  It  is  devel- 
oped to  be  a prestige,  a highly  desirable  organization 
to  be  a part  of.  There  is  a considerable  amount  of 
pride  in  having  achieved  the  opportunity  to  become  an 
NDI  technician  engineer  type. 

We  understand  that  very  few  that  achieve  that  level 
ever  leave  of  their  own  volition  or  are  forced  to  leave 
because  of  poor  conduct  in  carrying  out  their  tasks. 

This  is,  in  a brief  discussion,  one  of  the  things  that 
was  discussed  at  length  relative  to  one  type  of  method 
for  evaluating. 

UNIDENTIFIED  SPEAKER: 

I would  like  to  comment  on  that.  One  of  the  dumps  we 
had  done  yesterday  will  cite  a sample.  We  looked  at 
folks  who  had  high  time  experience  prior  to  going  into 
the  NDT  field,  and  those  folks  were  consistently  among 
the  poor  performers.  I am  talking  something  10  years  or 


more  prior  to  going  into  the  NDT  field.  So  that  with 
respect  to  our  sample  group  this  evidently  did  not  hold 
true . 

MR.  WOLFORD: 

The  question  was  that  in  their  discussion  group 
yesterday  they  reviewed  computer  printouts  that  showed 
that  people  with  high  time  experience  prior  to  going 
into  the  NDI  field  did  not  necessarily  perform  in  the 
NDI  field  in  the  top  capability. 

We  saw  some  of  those  printouts.  We  also  noted  that 
depending  on  one  discipline,  eddy  current  versus 
radiography,  that  it  was  pretty  random.  I don't  know 
that  we  could  really  say  that  we  saw  that  there  was 
consistently  poor  showing  on  those  that  had  other 
skills  prior  to  going  into  the  NDI  field. 

But  I am  only  relating  to  you  what  has  apparently  been 
very  successful  in  the  RAF  and  in  the  Canadian  Air 
Forces . 

Are  there  any  other  questions. 

MR.  BOISVERT: 
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II 


TRAINING 


TASK  GROUP  LEADERS: 

John  Dorgan 
C.  J.  Hellier 
V.  L.  Stokes 

JOHN  DORGAN,  Summary  on  Task  Group  Training:  We  had 
about  26  people  to  participate  in  the  three  sessions 
that  we  had.  It  was  a pretty  good  cross  section  between 
Air  Force,  Army,  Navy,  salesmen.  We  had  some  gentlemen 
from  Australia.  We  had  some  from  England.  So,  it  was  a 
pretty  good  cross  section. 

It  is  pretty  hard  to  stand  up  here  and  relate  what  26 
people  were  trying  to  communicate.  But  the  first  thing 
that  we  wanted  to  do  was  try  to  find  out  just  who  did 
we  want  to  train,  what  type  of  individual.  And  this 
more  or  less  overlaps  into  the  preceeding  presentation 
of  personnel. 

If  you  cannot  read  them,  they  are: 

Integrity 

Desire  to  be  in  NDI 

That  means  he  has  to  volumteer.  We  are  not  going  out  to 
solicit  people  to  be  in  NDI.  They  have  to  want  to  be  in 
the  field  in  order  to  work  in  it. 


Above  average.  What  this  means  is  his  past  history,  his 
past  experience  in  companies  or  in  military,  or  what- 
ever the  case  may  be,  he  had  to  be  an  above-average 
type  performer,  this  sort  of  thing. 


And  adaptability.  This  means  that  he  would  have  to  be 
able  to  physically,  mentally,  and  so  forth  to  accept 
the  responsibilities  of  the  areas  that  he  was  expected 


to  work 

in  such 

as  cold  , hot , 

, flight 

line,  in-shop. 

these  sor 

t of  things. 

So,  this 

was  the 

type  person 

that  we 

had  determined 

that  we  really  wanted  to  train  and  really  wanted  into 
the  field.  In  all  three  sessions  we  had  a considerable 
amount  of  discussion  about  not  so  much  as  to  the  time 
dedicated  to  the  training,  although  we  did  use  a base 
of  like  80  hours  one  time  for  radiography,  but  the 

amount  of  time  with  classroom  formal-type  training  

this  is  in  a lecture-type  environment  versus  a practi- 
cal hands-on- type  thing  where  you  are  actually  using 
the  equipment  with  an  inspector  or  another  person  of 
higher  knowledge  looking  over  your  shoulder  to  give  you 
instructions  in  the  particular  inspection  medias. 

You  can  see  the  first  phase.  The  first  phase  which  we 
considered  in  the  military,  or  like  the  Air  Force, 
would  be  considered  the  three  level.  And  in  industry  it 
would  be  the  level  one  certification.  That  was  class- 
room, 45  percent,  and  the  practical  hands-on,  55  per- 
cent . 

Now,  there  was  a little  bit  of  confusion  between  some 
of  the  participants  in  that,  for  instance,  reading 
radiographs,  was  that  considered  hands-on  or  was  it 
considered  formal  classroom  training.  So,  there  was 
somewhat  of  a split  opinion  there  as  to  which  one  it 
correlated  to. 

In  phase  two,  which  we  considered  in  the  air  Force  as 
being  five-level  training  but  in  the  industry,  level 
two  training,  that  correlated  to  35  percent  in  the 
classroom  environment  and  65  percent  for  hands-on 
training.  There  was  a lot  of  discussion  leading  up  to 
these  percentages,  that  being:  Where  do  you  want  the 
most  of  the  emphasis,  out  there  where  he  is  actually 
working  on  the  equipment,  where  you  make  your  money, 
where  you  had  the  reliability  in  him,  if  he  happened  to 
miss  a discontinuity  or  miss  a defect  that  you  want  him 
to  look  at,  if  you  want  him  to  be  a part  of  the  equip- 
ment so  that  he  wouldn't  really  understand  thermionic 
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emission  but  just  know  how  to  operate  the  equipment 
itself  versus  a deep  theory  and  understanding  of  what 
is  actually  going  on  inside  the  material. 

The  next  item  that  we  went  to  is:  How  is  effectivity  of 
training  measured?  There  were  three  items  that  we  came 
up  with,  one  being  the  peformance  on  the  job.  That  is 
that  when  an  individual  first  comes  into  your  function 
or  your  organization  what  you  do  is  that  you  inspect 
after  he  has  inspected  an  item.  That  means  that  he 
inspects  something  and  then  you  go  out  right  after  him 

and  see  what  he  has  told  you  is  correct  until  you  have 
built  some  kind  of  confidence  in  him. 

Examinations.  This  was  written  theory  as  well  as 
practical  examination,  somewhat  similar  to  the  "Have 
Cracks  will  Travel  Program." 

Evaluation  by  supervision.  A lot  of  this  whole  program 
in  training  and  in  what  is  expected  of  the  individual 
lies  right  on  the  integrity  and  the  enthusiasm,  spirit 
of  the  supervisor  himself.  You  know  yourself  that 
whatever  your  supervisor  expects  of  you  that  you  are 
going  to  try  to  rise  to  his  expectations.  So,  the  whole 
program,  regardless  of  what  it  reads  on  paper  and 
regardless  of  what  it  tells  us  to  do,  lies  right  in  the 
hands  of  the  supervisor  --  how  he  produces  or  controls 
his  own  program. 

Should  personnel  have  single  or  multiple  discipline 
capabilities?  Now  this  again  is  according  to 
circumstance  or  in  an  area  in  which  the  person  is 
expected  to  function.  In  an  i nd us t r y- typ e mass 
production-type  thing  we  concluded  that  it  would  be 
sort  of  silly  for  a manufacturer  or  an  organization  to 
train  somebody  in  all  the  functions  of  NDI  if  all  he  is 
expected  to  do  for  instance  is  sit  on  an  assembly  line 
and  read  items  going  through  a penet rant- type 
inspection.  This  would  be  considered  quite  a limited 
type  NDI  capability. 

Now,  on  a general  type  thing  where  a person  is  expected 
to  make  decisions  --  well,  excuse  me.  In  the  other 
method  he  wouldn't  make  the  decisions.  All  he  would  see 
is  probably  an  indication  and  that  he  would  call  some- 
body else  to  come  and  make  a decision  on  the  product. 
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In  the  general  sense  like  in  the  Army,  Navy,  Air  Force, 
and  NASA  some  of  these  other  functions  where  you  are 
expected  to  know  about  more  than  one  type  function  or 
make  a decision,  maybe  this  is  not  the  best  way  to 
inspect  something,  that  he  should  be  expected  to  know 
about  all  the  functions.  He  should  not  necessarily  be 
able  to  have  the  expertise  to  be  able  to  be  certified 
in  it  or  something  like  this  but  at  least  have  knowl- 
edge of  what  the  other  inspection  medias  have  or  how 
their  function  is  or  just  the  general  basic  idea  of  how 
they  operate. 

Should  refresher  courses  be  given?  Emphasis  should  be 
placed  on  upgrading  new  technician  developments  and  so 
forth  . 

We  ran  several  computer  runs  on  this  to  try  to  extract 
out  of  the  computer  some  information  on  capabilities 
versus  how  they  fared  in  the  "Have  Crack  Will  Travel 
Program."  We  found,  which  surprised  everybody,  quite  to 
the  contrary  that  in  many  cases  in  the  computer  -- 
notice  I said  in  the  computer  — it  stated  that  a 
person  had  999  hours  of  training  in  eddy  current  and  he 
fared  in  the  bottom  10  percent  of  the  program.  On  the 
other  hand  it  showed  that  a person  maybe  had  80  hours 
training  in  eddy  current  and  he  fared  in  the  top  10 
percent.  So,  I don't  myself  --  okay,  I am  going  to  put 
a personal  note  in  here.  I,  myself,  don't  believe  that 
this  is  at  all  accurate.  I think  that  on  the  form  that 
he  might  have  put  down  there  that  his  total  NDI  experi- 
ence and  not  necessarily  just  his  training  in  eddy 
current.  So,  therefore,  I tend  to  discredit  the  find- 
ings in  the  computer. 

Okay.  Now,  what  we  thought,  what  the  final  concensus 
was  was  that  if  you  have  a poor  performer,  you  might 
try  to  upgrade  him  once  or  twice  and  try  to  bring  him 
up  to  your  expectations.  But  after  once  or  twice  it  was 
kind  of  trying  to  put  good  money  against  bad  in  trying 
to  train  into  a person  some  type  of  integrity  or 
ability  that  he  evidently  had  no  interest  in  performing 
in  the  first  place. 
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The  main  type  of  refresher  courses  should  be  on  the 
line  of  new  techniques  that  are  developed  in  the  art, 
new  equipment,  maybe  some  new  findings  like  for 
instance  acoustic  emission  or  low  frequency  eddy 
currents,  something  along  this  line. 

Centralized  training  versus  mobile  training  units. 
There  are  two  items  here  that  we  found  that  were  more 
advantageous  to  have  a centralized  t r a i n i n g- 1 ype 
environment,  one  being  that  at  a centralized  training- 
type  environment  the  individual  was  given  the  greater 
opportunity  of  remaining  in  the  classroom  environment. 
He  was  not  interrupted  by  phone  calls,  higher  priori- 
ties that  his  supervisor  might  have  to  have  to  pull  him 
out  of  the  session  to  go  and  do  a particular  job,  this 
type  of  thing.  Plus  the  economics  of  it.  We  had  several 
gentlemen  in  the  session  that  had  experience  in  this 
line.  And  they  informed  us  that  it  was  more  economical 
to  send  the  people  to  the  centralized  training  rather 
than  having  them  travel  to  the  section  or  to  the  area 
of  the  shop,  laboratory,  whatever  to  do  the  training. 
There  was  a difference  of  about  $200  or  $100  a person. 

Can  training  kits  be  devised  to  provide  hands-on 
experience?  Yes,  if  complete  supplemental.  What  we 
meant  by  this  is  that  these  kits  --  which  we  took  them 
to  be  sound  on  slides.  If  some  people  are  not  familiar 
with  that  term  it  is  a tape  recorder  that  is  wired  up 
to  slide  projector.  I don't  really  know  the  function  of 
it.  But  the  tape  triggers  the  slide  projector  while  it 
is  giving  the  dialog.  It  shows  a slide  and  more  or  less 
is  a show-and-tell  type  thing.  Plus,  what  would  be 
containeed  in  this  kit  would  be  --  or  what  it  would  be 
geared  to  is  a piece  of  equipment  or  a particular 
inspection  media  such  as  ultrasonics.  It  would  give  the 
technique  setup  of  the  piece  of  equipment,  whatever  it 
may  be,  plus  a sample  that  you  are  to  inspect,  hands- 
on-type  thing.  And  it  will  show  pictures  of  the  sample 
and  where  you  should  place  the  transducer,  eddy  current 
probe  or  how  you  would  set  up  the  radiographic  tech- 
nique, this  type  of  thing.  And  it  would  be  somewhat 
more  understanding  than  just  possibly  reading  the  book 
because  you  would  have  the  equipment  and  support  data 
there.  So,  it  would  be  a little  bit  more  understanding 
than  just  a text  data  itself. 


How  much  theory  training  is  necessary?  Enough  to  under- 
stand the  mechanics  of  the  test. 


Now,  this  goes  hand  in  hand  with  a prior  question.  I 
believe  it  was  45  percent  and  55  percent.  There  were 
two  trains  of  thought  here,  one  being  how  possibly 
could  anybody  make  or  do  an  inspection  if  he  did  not 
have  the  complete  theory  as  to  what  was  happening 
between  the  part  itself.  Other  people  say,  why  clutter 
his  mind  up  with  this  type  of  thinking  when  all  we  want 
him  to  do  is  find  a discontinuity  or  defect. 

Should  training  be  tailored  to  achieve  certification? 
This  was  quite  emotional  with  me  and  everybody  else.  We 
concluded  that  training  should  not  be  geared  to  just 
achieve  certification.  Certification  should  be  the 
minimal  requirement  for  doing  or  functioning  or  achiev- 
ing any  one  particular  inspection.  Training  should  be 
geared  to  receive  the  optimum  effectiveness  in  inspec- 
tion capability.  I don't  know  what  else  to  say. 


Should  the  trainer  be  certified?  There  should  be  a 
system  to  establish  a level  of  qualification  and/or 
certification  for  NDI  trainers.  Some  years  ago,  maybe 
10  or  15  years  ago,  when  the  first  generation  of  really 
informed  experts  in  NDI  were  coming  up,  there  was  not 
anybody  to  do  any  certifying  because  who  knew  enough 
than  those  type  people  that  could  certify.  I have  no 
idea  . 

But  now  I estimate  that  we  are  probably  up  in  our 
second  or  third  generation  of  NDI  inspectors.  This 
should  be  a prerequisite.  We  should  not  put  people  in  a 
classroom-type  environment  of  people  that  have  little 
or  no  experience  in  NDI  other  than  what  they  have  read 
out  of  a book.  There  are  a lot  of  things  that  you  read 
out  of  a book. 

But  there  is  so  much  more  that  you  learn  out  of  prac- 
tical experience.  And  this  should  be  the  situation. 

Should  management/supervision  be  NDI  oriented?  Yes. 
Suggest  possible  three-day  appreciation/t ami  1 iari zation 
course.  I personally  have  been  put  in  situations  where 
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an  individual  that  had  somewhat  more  authority  than  I 
had  had  gone  through  an  orientation  course  where  it  was 
possibly  only  three  days  long  and  stuff  like  this.  And 
because  of  the  virtue  of  his  position  he  naturally 
thought  he  knew  more  than  I did.  What  I would  like  to 
suggest  on  this  with  the  concensus  of  the  forum  itself 
was  to  not  only  orientate  people  that  weren't  directly 
involved  in  NDI  but  are  on  a management  level  to  be 
familiar  with  NDI,  but  not  only  terl  them  of  the  capa- 
bilities of  NDI  but  also  tell  them  of  the  shortcoming, 
what  a particular  inspection  media  cannot  do  as  well  as 
what  it  can  do.  So,  you  don't  get  things  like:  Well, 
you  didn't  find  it,  why  don't  you  X-ray  it,  you  know, 
this  type  thing  because  any  one  of  these  type  circum- 
stances or  any  one  type  inspection  media  is  not  a 
cure-all  for  all  situations. 

Comparison  of  performance  versus  training  hours.  Now, 
this  was  the  data  which  we  received  out  of  the  com- 
puter. Based  on  computer  supplied  data  we  found  that 
performance  did  not  improve  with  increased  hours  of 
training.  I think  I already  spoke  on  this.  Performance 
did  not  correlate  with  the  level  of  formal  education. 
We  found  that  in  the  lower  and  top  10  percent  that 
formal  education  outside  of  NDI  — there  was  no  cor- 
relation with  the  amount  of  formal  education  versus 
their  performance  in  NDI.  There  was  not  anything  in  the 
computer  to  tell  us  what  type  of  education  or  in  what 
subject  or  what  they  majored  in  or  anything  like  this. 

Performance  did  not  correlate  with  the  amount  of  NDI 
experience.  There  again  it  was  already  mentioned  that 
people  with  more  than  10  years'  experience  were  in  the 
top  10  percent  and  in  the  bottom  10  percent  more 
evidently.  And  those  with  less  than  10  years  were 
prominent  in  the  top  10  percent.  Again,  people  with 
more  than  10  years'  experience  were  in  the  top  10 
percent . 

In  fact,  it  was  a primary  factor  that  that  is  usually 
where  they  were  at.  People  with  less  than  two  years' 
experience  fell  by  and  large  in  the  bottom  10  percent. 

Do  you  all  know  what  I am  talking  about? 
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Again,  performance  did  not  correlate  with  the  amount  of 
NDI  experience.  Those  with  less  than  two  years'  experi- 
ence fared  better  than  those  with  over  10  years' 
experience  . 

Age  had  no  bearing  on  peformance.  Here  again  just  like 
the  formal  training,  the  ages  were  scattered  from  one 
range  from  the  young  to  the  old  in  the  top  10  percent 
as  well  as  the  bottom  10  percent. 

Recommendations:  Bernie  asked  for  some  recommendations 
that  we  might  give  to  him  or  suggest  to  him  in  regards 

to  the  fact  that  if  he  should  run in  the  terms  of 

training,  if  he  should  run  this  program  again,  what 
were  some  of  the  things  that  we  suggest  could  be 
changed . 

Evaluate  existing  training  programs.  What  we  meant  by 
this  is  again  the  integrity  and  the  experience  level  of 
the  instructors  as  well  as  when  you  go  into  a labora- 
tory or  a shop,  whatever  it  may  be  called,  that  you 
check  the  credentials  of  the  supervisor  even  if  he  is 
tested  on  the  reliability  program  or  not,  that  you  go 
in  and  test  the  credentials  of  the  supervisor  to  find 
out  if  he  expects  the  best  quality  out  of  his  people 
versus  let  them  do  it  as  long  as  the  paper  work  is 
filled  out,  that  we  have  an  integrity-type  situation 
where  the  supervisor  will  ensure  that  his  people  are 
doing  the  best  job. 

Evaluate  qualifications  of  the  instructor. 

Evaluate  training  materials,  tex tbooks  , manuals , aids, 
so  forth.  This  was  meant  by  the  textbooks,  manuals  and 
the  aids,  etc.,  that  are  in  the  "Have  Cracks  Will 
Travel  Program." 

Establish  a system  for  NDI  trainee  selection.  I think 
the  personnel  people  have  already  covered  this.  Plus, 
on  our  very  first  slide  we  showed  that  these  are  the 
people  that  we  would  like  to  train  and  this  is  how  we 
would  like  to  train  them. 

More  detailed  input  data  to  the  computer.  When  we  fill 
out  this  form,  we  are  not  saying  more  pieces  of  infor- 
mation but  more  explicit  pieces  of  information.  Such 


as,  when  it  says  on  a particular  form  like  for  eddy 
current,  specimen  A,  or  whatever  it  was,  that  he  put  on 
there  eddy  current  training  rather  than  NDI  training 
because  otherwise  there  is  no  correlation.  He  might 
have  had  10,000  hours  training  in  radiography.  I don't 
know  what  value  that  would  be  in  eddy  current.  That  way 
we  would  have  more  correlation  to  find  out  just  how 
much  training  is  effective  in  any  one  particular  in- 
spection media. 

Okay.  Are  there  any  questions  on  what  we  have  covered? 
UNIDENTIFIED  SPEAKER: 

You  kept  saying  supervisor.  Did  you  mean  instructor? 

MR.  DORGAN: 


No.  In  some  cases  when  the 
I say  person  because  we 
in  the  Air  Force  --  but 
working  environment,  he 
like  O.J.T.  even  if  it 
of  the  emphasis  is  put 


person  or  the  individual  -- 
have  got  a lot  of  ladies  in  NDI 
when  that  individual  goes  to  a 
is  still  on  a type  of  training 
is  just  orientation.  And  a lot 
on  what  the  supervisor  expects 


of  his  people  in  the  way  of  a learning  environment. 
UNIDENTIFIED  SPEAKER: 

I would  like  to  clarify  a point  brought  out  about  the 
question  of  the  amount  of  experience  in  NDI. 

MR.  DORGAN: 


He  wants  to  clear 
experience  in  NDI. 


up  a point  in  the  question  about 


UNIDENTIFIED  SPEAKER: 


computer  run,  it  showed  that 


According  to  the 

people  with  less  than  two  years'  experience  in 
acutally  did  as  well  if  not  better  than  those  with 
than  10.  I would  like  to  refer  back  to  the  first 
sion  when  talking  about  personnel.  Is  that  the 
that  the  gentleman  from  the  back  of  the  room 
referring  to  when  he  talked  about  in  excess  of  10  years 
experience  in  other  fields? 


those 

NDI 

more 

ses- 

data 

was 
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The  question  was  on  the  correlation  between  the  experi- 
ence, was  it  NDI , or  was  it  in  other  fields?  This 
correlation  was  the  prior  experience  of  other  fields  of 
more  than  10  years  and  here  in  the  NDI  field.  Those 
people  were  in  the  low  performance  area.  The  other 
correlation  we  did  not  look  at. 

UNIDENTIFIED  SPEAKER: 

Well,  the  reason  I am  confused  on  this  point  was 
because  in  the  session  I was  in,  personnel,  we  tried  to 
get  some  figures  as  to  how  well  a person  performed 
during  the  first  four  years  of  NDI  if  that  was  his 
first  job,  right  off  the  street,  compared  to  the  man 
that  had  been  in  some  type  of  industry  job  or  aircraft 
maintenance  job  for  a period  of  four  or  five  years  and 
then  went  into  it.  Data  was  not  available.  Now  you  are 
saying  that  those  with  10  years  did  perform. 

UNIDENTIFIED  SPEAKER: 

Yesterday  when  I went  into  Personnel  they  had  a differ- 
ent listing  and  not  a complete  a listing  as  some  of  the 
other  task  groups.  There  is  more  information  available 
than  was  in  that  task  group. 

UNIDENTIFIED  SPEAKER: 

Can  you  clarify  something  for  me,  John,  please? 

What  I would  like  to  ask  is:  My  understanding  of  the 
data  of  reliability  against  experience  --  my  under- 
standing of  the  situation  on  this  data  was  that 
reliability  in  finding  cracks  against  number  of  years 
of  experience  and  against  number  of  hours  of  formal  NDI 
training  was  that  it  was  a scatter,  that  you  could  get 
one  guy  who  had  10  years'  experience  in  NDI  who  might 
not  be  very  good  at  finding  cracks  whereas  another  one 
with  10  years  could  be  very  good.  A guy  with  two  years' 
experience  in  NDI  could  be  very  good  or  he  could  be  not 
so  good.  There  was  a complete  scatter. 

Is  this  true? 
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MR.  DORGAN: 


We  found  that  it 
than  two  years' 


was  dominant  that 
experience  fared 


than  those  that  had  over  10  years' 
Is  that  true,  Bernie? 


the  people  with  less 
a whole  lot  better 
experience . 


MR.  BOISVERT: 

This  is  a rough  look.  We  are  going  to  have  to  clarify 
this.  But  some  of  the  printouts  that  they  had  indicated 
that  the  higher  performers  were  two  years  or  less. 

MR.  DORGAN: 

That  is  the  way  the  trend  looks  right  now. 

UNIDENTIFIED  SPEAKER: 

Because  the  comment  is  that  if  the  case  is  as  I put  it, 
then  the  fact  is  that  you  haven't  got  a selection  pro- 
cedure and  that  your  selection  from  a random  population 
will  explain  that  scatter. 

UNIDENTIFIED  SPEAKER: 

Bernie,  you  need  to  be  able  to  distinguish  between  10 
years  of  experience  and  one  year  of  experience  10 
times  . 

MR.  BOISVERT: 

Yes.  That  is  a good  point. 

UNIDENTIFIED  SPEAKER: 

A word  of  caution  about  your  terminology  correlation. 
You  mentioned  the  performance  didn't  correlate  with  the 
level  of  formal  education  or  NDI  experience. 

Are  you  assumng  a linear  regression  model?  If  so,  why? 
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MR.  DORGAN: 


Chuck 

MR.  HELLIER: 

No,  we  were  not  assuming  a linear  regression  model.  It 
was  just  extracting  the  bits  of  information  that  we  had 
on  the  computer  run-off  sheets. 

MR.  BOISVERT: 

They  need  the  question  repeated. 

UNIDENTIFIED  SPEAKER: 

My  question  was:  Did  he  assume  a linear  regression 
model?  If  so,  why? 

MR.  DORGAN: 

No.  There  was  no  linear. 

Okay.  Tliat  is  all  the  time  we  have.  Thank  you  very 
much.  I would  really  like  to  thank  everybody  that  was 
in  our  session  for  the  cooperation  and  the  really  good 
discussions  that  we  had.  Thank  you. 
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OPERATIONS 


TASK  GROUP  LEADERS: 

E.  0.  Lomerson 
E.  C.  Gustely 
R.  J.  Meyer 

MR.  E.  0.  LOMERSON,  Summary  on  Operations:  Bernie,  I 
really  couldn't  have  picked  three  more  impartial 
people.  Ed  Gustely,  Richard  Meyer,  and  myself  have  no 
background  at  all  in  the  service  portion  of  this.  So, 
most  of  the  stuff  we  are  giving  you  are  the  comments 
that  we  had  gotten  from  the  18  or  so  people  from  the 
various  walks  of  life  here,  the  field,  the  depot,  the 
Army,  the  Navy,  and  the  Royal  Air  Force,  plus  industry. 

The  first  thing  we  tried  to  do  is  to  figure  out  just 
what  the  objective  was.  The  operation  didn't  give  us 
too  much,  unfortunately.  I don't  think  the  questions 
were  that  enlightening  for  use  to  figure  out  what  we 
wanted  to  do.  So  we  came  up  with  the  objective  of  What 
Management  tools  — and  I guess  the  Air  Force  and 
military  have  different  terminologies  than  I used,  but 
whatever  their  terminology  would  be  --  What  Management 
Tools  Can  be  Used  To  Improve  NDI  Effectiveness.  So, 
this  is  what  we  tried  to  go  after. 

During  the  various  discussions  we  aimed  in  on  problem 
identification.  We  have  got  a list  of  what  we  feel  were 
the  problem  identifications.  Now,  the  first  one  is 
Inadequate  Recognition  of  the  NDI  Personnel.  Obviously, 
this  is  a beautiful  motherhood  statement  specifically 
for  an  NDT  gathering  like  this.  But  this  specific 
statement  came  up  many  times,  and  it  was  felt  to  be 
important  enough  to  put  as  first. 

Now  the  organization  in  the  air  force  is  such  that  the 
way  the  commands  come  down,  the  NDI  inspection  appears 
to  be  able  to  be  overridden  or  enough  pressure  is  put 
on  that  they  will  maybe  inspect  a little  bit  different 
than  if  they  had  a little  bit  more  of  authority. 

Unrealistic  work  Schedules,  being  number  two,  was 
discussed  several  times.  The  work  schedules  coming  down 
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as  supplied  by  the  depot  were  such  that  it  was  feasibly 
impossible  to  do  the  inspection  in  the  time  allotted. 


impo 
So, 
had  to 


be 


f ore , 

- various 

types 

of 

shortcuts , 

et 

cetera , 

appl 

ied  . 

Not 

Exclusively  Assigned  to  NDI. 

In 

several 

was 

mentioned 

that 

NDT 

personnel 

were 

pulled 

Personnel 
places  it 

from  sheet  metal  or  other  areas  and  that  the  NDT  was 
more  or  less  a second-area-type  operation.  In  other 
words,  when  they  needed  something  done,  they  pulled 
them  from  the  other  area.  We  will  get  back  to  that  a 
little  bit  later. 


The  fourth  is  Nonqualified  Personnel  Assigned  To 
Perform  NDT  Inspections.  Now,  I think  this  was  probably 
the  most  pushed  requirement  or  problem  identification 
here.  Everyone  in  all  three  groups  said  that  the 
personnel  doing  the  NDT  inspections  were  very  limited 
in  their  certification,  qualification,  or  in  their 
backg  ro  und . 

The  fifth  is  NDT  Technical  --  and  when  we  say  NDT 
Technical,  we  mean  the  depot  — Personnel  Lacks  the 
Authority  To  Monitor  NDT  Field  Operations. 

It  was  pointed  out  that  many  times  the  fields  are  doing 
one  field  operation  where  another  does  not  do  the 
inspection  anywhere  near  the  same.  Some  are  more,  let's 
say,  reliable  or  sensitive  than  in  other  places. 


And,  finally,  the  NDT  techniques  — these  are  the 
T.O.'s  issued  by  the  Air  Force.  I guess  it's  a bulletin 
by  the  Navy  — provided  to  the  field  are  not  necessar- 
ily applicable  under  field  environments.  Techniques  can 
be  set  up  in  the  labs  and  when  they  get  to  the  field, 
they  cannot  be  done  under  actual  inspection  conditions. 
This  is  both  the  technique  and  with  the  existing  equip- 
ment in  the  field. 

As  a result  of  these  requirements  or  problem  identifi- 
cations we  came  up  with  some  of  the  recommendations 
that  were  felt  to  be  important. 

One  was  that  NDI  personnel  should  report  directly  to 
the  chief  of  maintenance.  I guess  in  the  Air  Force 
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right  now  they  are  reporting  to  the  fabrication  shop. 
The  fabrication  shop,  since  they  are  required  to  get 
these  planes  turned  around  and  pushed  out  as  rapidly  as 
possible  tend  as  we  understand  it  to  put  considerable 
pressure  on  the  NDI  people  and  thus  possibly  overrule 
certain  types  of  inspections  or  persuade  the  inspectors 
to  accept  something  that  may  not  be  correct. 

Now,  obviously,  going  back  to  the  problem  of  identi- 
fication where  we  said  inadequate  recognition  given  to 
the  NDT  inspectors,  this  change  in  organization  will 
not  in  itself  give  you  that  recognition.  We  feel  the 
NDT  personnel  have  to  become  knowledgeable,  very  strong 
in  the  NDT  inspection.  And  when  they  call  something 
wrong,  it  is  wrong.  It  isn't  that  well  it  might  be  or 
it  might  not  be.  They  have  got  to  put  theirselves  on 
the  line  and  say  this  is  it  and  we  will  back  it  up  to 
the  hilt.  Sometimes,  obviously,  they  may  be  wrong.  But 
what  you  have  got  to  do  is  get  confidence  of  the 
officers  in  charge. 


NDI  Time  should  be  determined  by  designated  depot  NDT 
engineers.  In  other  words,  if  a T.O.  comes  out  and 
says:  Look,  we  have  got  to  re-inspect  this  part  and  it 
has  got  to  be  turned  around  in  two  days,  it  means  that 
under  field  conditions  it  can  be  done.  The  concensus  a 
lot  of  times  was  that  some  of  these  T.O.'s  are  coming 
down  indicating  times  that  are  unrealistic.  Therefore, 
to  meet  the  times  they  are  taking  shortcuts,  doing 
sampling,  or  in  some  way  getting  those  planes  back  on 
the  line  because  this  is  the  requirement  they  are  there 
for.  Now,  we  recognize  that  there  are  other  problems 
that  supervision  has  to  take  care  of  to  make  sure  the 
people  are  working.  But  we  are  talking  about  under  the 
conditions  that  it  is  truly  unrealistic. 

The  third  item  is  NDT  Personnel  Should  Be  Assigned 
Primarily  To  NDT  Activity.  It  is  not  the  habit,  or  it 
is  not  reasonable  to  assume  that  a person  that  has 
responsibility  in  other  areas  and  an  NDT  as  a secondary 
will  be  capable  of  being  proficient  on  all  of  the  NDT 
techniques . 


It  would  be  better  to  have  fewer  NDT  people  with  good 
strong  NDT  training  and  using  this  training  all  the 


time  rather  than  a large  body  of  people  to  throw  in 
that  may  not  be  being  used  from  month  to  month. 

NDT  Personnel  Should  Be  Certified  In  Each  Method  By 
Testing  Prior  To  Performing  NDI . Again,  this  is  what  we 
mentioned  before  as  being  the  major  problem  identified, 
certification.  As  was  pointed  out  before,  we  were  not 
interested  in  how  you  certify  him,  just  to  make  sure 
that  they  are  capable  of  being  certified  and  doing  the 
job  that  is  intended.  This  certification  is  important 
on  both  the  civilian  and  the  military  personnel.  It 
should  be  to  something  like  410  or  equivalent. 

The  Depot  Personnel  Should  Be  given  Authority  To 
Monitor  NDT  Field  Operations  and  Assure  Corrective 
Action.  Now,  under  this  particular  case  it  is  felt  that 
the  depot  has  the  prime  responsibility.  So,  therefore, 
you  treat  them  the  same  as  the  field  should  be  treated 
or  not  treated  but  should  be  controlled  similar  to  a 
vendor  from  a prime,  whereas  to  assure  that  they  are 
getting  the  same  kind  of  inspection  from  one  field 
operation  to  another,  in  addition  that  the  depot  should 
be  writing  the  technique  and  should  be  fairly  uniform 
and  assure  that  the  techniques  are  correct. 

The  last  items  is  somewhat  under  the  same  area  as  the 
previous  one.  All  NDT  Technique  Sheets  Should  Be 
written  And/Or  Reviewed  Or  Approved  By  Designated  NDT 
Depot  Engineers.  It  has  been  indicated  in  one  or  two  of 
these  meetings  that  in  most  cases  this  is  being  done, 
however,  as  sometimes  flight  officers,  etc.,  without 
any  NDT  experience  may  demand  or  require  certain 
inspections  and  go  around  the  NDT  engineer  in  the  depot 
and  these  types  of  inspections  can  get  out  of  hand  from 
time  to  time. 

Now,  for  the  depot  we  did  put  a couple  items  down  that 
we  thought  we  should  just  mention  in  passing,  that 
their  responsibility  should  be  to  provide  a simple, 
well  defined  T.O.  or  technique,  whatever  you  want  to 
call  it.  They  should  assure  that  the  field  has  the 
proper  equipment  and  personnel  before  issuing  such  a 
T.O.  They  should  supply  special  tools,  fixtures, 
calibration  blocks,  et  cetera  when  they  develop  or 
write  the  T.O.  They  should  provide  clear,  'straight- 
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forward  standards  that  can  be  interpreted  relatively 
easily.  They  should  check  techniques  under  field 
conditions,  actually  at  the  base.  I think  the  Navy  has 
indicated  they  have  a way  of  sending  their  T.O.  or 
bulletin  out  to  be  approved  by  some  field  operation 
before  it  gets  issued.  They  should  investigate  and 
specify  a realistic  time  for  inspection. 

They  should  provide  training  assistance  to  the  field. 
And,  finally,  they  should  assure  that  all  field 
installations  are  doing  the  same  inspection  whether 
this  is  by  monitoring  the  field  operation.  I don't  want 
to  get  into  politics  in  here,  but  what  they  should  be 
able  to  do  is  get  out  into  the  field  and  actually  do  an 
audit  to  make  sure  that  the  field  is  doing  the  inspec- 
tion correctly  and  if  not,  have  authority  to  take  and 
implement  corrective  action. 

Relative  to  the  questions  listed  under  Section  3 
covering  time  and  motion  studies,  work  and  rest 
periods,  task  sizes,  et  cetera,  it  was  concluded  that 
each  job  was  unique  and  should  rest  on  its  own  merits. 
In  other  words,  we  could  not  define  specifically  the 
proper  task  size  or  the  rest  periods,  how  long  it 
should  take  before  you  take  a break  on  the  thing.  The 
NDT  supervisor  should  be  thoroughly  knowledgeable  in 

the  applicable  NDT  methods  and  it  is  his  judgement  

it  is  up  to  him  to  decide  this.  This  is  what  he  should 
be  paid  for  and  should  control.  However,  it  should  be 
recognized  that  the  task  time  related  to  a specific  NDT 
inspection  is  important  both  to  obtain  some  type  of  an 
efficiency  but  more  importantly  to  establish  a 
realistic  schedule. 

So,  therefore,  Bernie,  I am  sorry  we  could  not  give  you 
specific  answers  to  your  questions. 

I think  that  is  it  unless  there  are  any  questions. 

UNIDENTIFIED  SPEAKER:  Yes. 

The  -36  manual  and  lots  of  T.O.'s  are  written  by  the 
contractor  on  his  product.  I still  think  that  that 
should  be  encouraged  here  and  not  just  turn  it  over  to 
the  Air  Force. 
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MR.  LOMERSON: 


I 


We  recognize  this,  but  we  said  that  the  depot  should 

write  or  review  and  approve. 

UNIDENTIFIED  SPEAKER: 

Well,  like  I said,  the  contractors  write  and  issue 
through  the  APPRO  and  the  Air  Force  SPO  System. 

Many  T.O.'s  don't  go  that  route.  However,  those 

documents  are  taken  out  and  actually  demonstrated  on  a 
vehicle  where  you  measure  the  times  and  how  long  it 
takes  to  get  to  the  article  and  to  make  sure  it  works. 
That  system  I don't  think  should  be  negated  and 

suddenly  everything  goes  through  the  depot. 

MR.  LOMERSON: 

We  felt  that  the  depot  was  more  or  less  the  prime  for 
the  field.  But  we  recognized  that  a lot  of  this  is  done 
by  industry.  We  felt  that  if  the  depot  is  going  to  have 
the  responsibility  to  assure  adequate  inspection  that 
they  should  also  have  a crack  at  the  technique.  Now, 
whether  it  is  a functionary  approval  or  otherwise  at 
least  they  should  be  there  because  they  may  be  out  in 
the  field  assisting  the  inspectors  there  to  implement 
it . 

Do  you  see  what  I mean? 

UNIDENTIFIED  SPEAKER:  Yes. 

UNIDENTIFIED  SPEAKER: 

I just  wanted  to  say  that  the  depot  NDI  engineer  does 
have  responsibility  for  the  -36's  regardless  of  who 
writes  them.  Even  though  a contractor  will  write  them, 
the  depot  has  got  to  have  responsibility  for  publica- 
tion and  implementation. 

MR.  LOMERSON: 

It  was  informed  to  us  that  they  could  go  around  them, 
that  they  have  the  responsibility  but  it  isn't  a 
straight  line-type  of  responsibility.  In  other  words. 
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not  all  T,0.'s  must  go  through  them,  just  as  Ed  Caustin 
just  indicated. 

Am  I right? 

UNIDENTIFIED  SPEAKER: 

They  may  go  by  them,  but  they  are  not  supposed  to. 

MR.  LOMERSON? 

This  is  the  reason  we  pointed  this  out. 

UNIDENTIFIED  SPEAKER: 

The  depot  is  not  in  the  loop  of  approval  in  the 
development  of  the  T.O.  The  T.O.  is  developed  and  is 
supposed  to  be  validated  by  the  contractor  on  the 
particular  model  of  the  aircraft.  If  in  the  field  they 
find  that  a particular  technique  is  not  correct,  the 
problem  is  that  there  is  no  feedback  between  the  field 
and  the  contractor  to  get  the  T.O.  changed. 

There  is  a missing  gap  in  the  system  there  that  causes 
delays  in  remedying  problems  in  the  T.O. 

MR.  LOMERSON: 

Would  you  repeat  the  question? 

UNIDENTIFIED  SPEAKER: 

I was  saying  that  the  contractor  develops  the  T.O.  and 
is  supposed  to  validate  the  techniques  that  are 
identified  in  the  T.O.  One  of  the  problems  that  exists 
is  if  in  the  field  operation  at  some  point  you  find 
that  the  T.O.  is  impractical  from  a field  standpoint, 
the  method  of  getting  the  T.O.  changed  has  a gap  in  it. 
And  this  causes  some  heartburn  I am  sure,  between  the 
contractor  and  the  NDI  personnel,  the  director  of 
maintenance,  or  whoever  in  the  field.  And  this  is  where 
the  problem,  I believe,  needs  to  be  reviewed  and  some 
corrective  action  taken. 


MR.  LOMERSON: 


Thank  you. 


UNIDENTIFIED  SPEAKER:  My  question  is  really  directed 
at  Bernie. 

In  these  presentations  --  we  have  not  heard  them  all 
yet.  But  it  seems  like  there  is  going  to  be  a great 
deal  of  overlapping  like,  training  talks  about  certi- 
fication, operations  has  talked  about  certification  and 
training,  and  we  wonder  whose  recommendations  should  be 
the  final  recommendations.  You  are  going  to  have 
possibly  conflicting  recommendations.  And  I am  just 
curious  as  to  how  you  are  going  to  handle  that. 

MR.  BOISVERT: 

We  like  these  conflicting  recommendations  primarily 
because  it  gives  us  some  options.  We  may  merge  the  two 
together  and  reach  a halfway  point.  But  we  do  want  or 
we  encourage  conflicting  recommendations.  This  is 
really  why  we  are  here.  I do  want  to  just  point  out  to 
both  Ed  Caustin  and  Roy  Wolford  according  to  AFPI71-159 
and  Mil  Specification,  MIL-M-38780  it  says  Ttiou  shalt 
submit  that  Tech.  Order  manual  to  the  depot.  And  if  we 
do  not  like  it,  we  don't  buy  it  off.  And  we  have  got 
the  SPO  ASD  under  our  thumb.  We  say:  Hey,  that's  not 
acceptable  to  us.  We  say,  do  it  over  until  we  like  it. 
We  also  have  a feed  back  information.  It  you  look  in 
Chapter  1 of  our  Tech  Order  33B-1-1  when  John  Dorgan 
wanted  his  people  to  come  up  with  a better  method  of 
doing  an  inspection  or  come  up  with  a part  that  could 
be  inspected  with  NDT,  they  fill  out  a form  and  it  goes 
into  the  depot.  If  you  are  still  under  contract  and,  in 
some  cases  Elmer  has  contacted  you  through  DSM  to  say, 
hey,  Mr.  Wolford,  develop  us  a procedure  around  this. 
We  do  have  a feedback  system.  It  doesn't  always  work. 


but,  hey,  it  is  there, 
but  we  have  a system. 


It  needs  sharpening  I will  admit 


UNIDENTIFIED  SPEAKER: 


I was  wondering  whether  it  is  universe 
industry  that  NDI  personnel  report 
management  rather  than  to  a production 
some  type. 


illy  true 

in 

directly 

to 

section 

of 
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MR.  LOMERSON: 


Ed,  why  don't  you  take  that. 

MR.  CAUSTIN: 

In  industry,  usually  they  report  to  quality  control, 
which  does  not  report  to  production  but  on  the  same 
level  as  production  or  engineering  or  any  of  the  other 
major  disciplines.  We  have  found  this  not  to  be  true  in 
the  Air  Force.  Of  course,  the  Air  Force  or  the  Navy, 
the  military  always  insists  upon  us  in  industry  to  have 
that  type  of  organization.  They  recognize  what  is  a 
good  organization,  but  they  don't  want  to  do  it. 

MR.  LOMERSON: 

They  recognize  that  they  need  that  qualification,  too. 
UNIDENTIFIED  SPEAKER: 

Ed  Caustin  builds  military  airplanes.  I just  wanted  to 
indicate  that  tor  commercial  aircraft  the  Federal 
Aviation  Regulations  also  require  that  the  inspection 
be  separated  from  the  manufacturing  operations. 

MR.  LOMERSON: 

Are  there  any  other  questions? 

Thank  you. 


IV 


CERTIFICATION 


TASK  GROUP 

LEADERS: 

J.  A. 

Moore 

G.  C. 

Wheeler 

S.  A. 

Wenk 

MR.  G.  C.  WHEELER,  Summary  on  Certification:  Well,  we 
did  not  address  ourselves  directly  at  any  time  to  the 
specific  questions.  Although,  after  we  got  done  I made 
some  effort  to  try  and  pull  together  answers  relating 
to  the  questions  that  Bernie  and  Bill  asked.  We  didn't 
really  have  to  because  the  first  session  started  out  so 
strong.  And  we  just  went  from  there.  It  was  quite 
apparent  that  everyone  had  a firmly  fixed  idea  about 
certification  and  whether  it  was  good  or  bad.  With  my 
particular  bias  I was  glad  to  find  that  everybody  was 
on  my  side. 

I think  the  reason  that  I was  nominated  to  stand  up 
here  was  that  I make  a poor  target  in  case  of  tomatoes. 

I have  never  seen  so  much  unanimity  on  certification 
before.  What  I am  going  to  do  is  give  you  the  punch 
line  first  because  as  I said  there  was  complete 
unanimity  that  certification  would  greatly  improve  NDI 
reliability. 

The  difficulty  was  to  get  people  to  try  to  quantify  why 
they  felt  that  way.  it  is  a highly  emotional  issue. 
Everyone  was  completely  in  agreement,  but  we  really 
couldn't  get  any  quantification  for  a while. 

Ultimately,  we  asked  the  computer  some  questions.  And 
specifically  what  we  asked  was:  What  proportion  of  the 
upper  25  percent  and  the  upper  10  percent  in  the  POD's, 
what  proportion  of  these  people  was  made  up  of  Base 
11 's  the  people  who  had  some  certification.  And  what  we 
found  was  this:  The  Base  11  's  made  up  about  eight 
percent  of  the  total  of  those  who  were  tested.  But  37 
percent  of  the  top  quartile  in  the  POD's  — 
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UNIDENTIFIED  SPEAKER: 


Can  I interrupt  a minute?  Could  you  tell  us  what  Base 
11 's  are? 

MR.  WHEELER: 

Base  11 's  are  that  one  group  that  were  certified. 

Thirty-seven  percent  of  that  top  quartile  was  made  up 
of  those  people.  I think  we  should  not  have  questions 
during  the  talk  in  order  to  get  the  thing  through.  I 

will  answer  questions  later. 

UNIDENTIFIED  SPEAKER: 

I would  like  to  ask  you  what  you  mean  by  certification 
of  existing  — 

MR.  WHEELER: 

Let  me  answer  that  later  because  that  is  a long  answer. 

It  seems  quite  clear  from  the  data  that  there  is  an 
effect  of  certification  on  how  well  people  perform. 

There  are  some  specific  glitches,  I think,  in  the 

computer.  We  found  things  like  people  with  900  hours  of 
training,  and  they  hadn't  been  born  yet.  There  are  some 
glitches  I think. 

These  averages  though  kind  of  spread  out  some  of  the 
glitches  and  let  you  look  at  the  general  case  rather 
than  the  specific  detail.  If  anyone  wants  a specific 
detail,  I can  still  dig  out  the  data. 

Some  of  the  questions  that  were  asked  by  Bernie  and 
Bill  and  which  one  can  always  ask  is:  How  can  you  do 
the  certification.  Now,  this  gets  to  the  point  of  being 
more  of  an  opinion  because  there  is  not  an  awful  lot  of 
data  on  that  kind  of  question  in  the  computer.  So,  what 
you  are  getting  now  are  opinions  that  I think  represent 
the  opinions  in  general  of  the  groups  that  we  talked 
to . 

The  general  opinion  seemed  to  be  that  the  certification 
should  be  based  on  exams.  You  can  ask  yourself,  how  do 
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I know  that  certain  material  has  been  covered  unless 
you  have  an  examination. 


The  general  feeling  also  was  that  there  was  a problem 
with  all  kinds  of  qualification  activities  if  they  are 
not  done  by  someone  who  is  a recognized  expert.  Now, 
you  can  ask  what  is  a recognized  expert.  We  left  it 
purposely  vague  because  there  are  a lot  of  definitions. 
I think  you  are  aware  that  there  are  places  to  go  for 
centralized  training  where  at  least  you  can  say:  I got 
this  body  of  knowledge.  It  is  a documentable  body  of 
knowledge.  Certain  questions  were  asked.  I know  the 
answers  to  those  questions. 

One  such  place,  of  course,  is  the  ASNT  Examinations. 
So,  there  are  ways  to  get  recognition  as  an  expert. 

Another  problem  that  seemed  to  be  very  common  is:  How 
should  you  administer  this  qualification  certification 
program.  And  again  we  resorted  to  this  generality  that 
it  ought  to  be  done  by  recognized  experts.  Clearly,  it 
doesn't  work  very  well  if  you  are  dealing  with  a boss 
who  doesn't  know  the  first  thing  about  NDI . 

And  then  there  was  a question  of  who  should  be  required 
to  certify.  Now,  we  thought  that  this  described  it 
pretty  well.  We  thought  certification  should  be 
required  of  anyone  who  was  practicing  NDI  on  flying 
objects . 

I think  that  in  --  I didn't  hear  any  objections  to  this 
in  any  of  the  sessions. 

We  also  felt  that  the  certifications  should  embody  some 
of  the  things  that  have  been  embodied  in  SNT-TC-1A 
because  that  is  an  easy  way  to  describe  that  you  should 
have  some  kind  of  an  examination  on  the  theoretical 
aspects  of  the  method,  something  about  the  specific 
equipment  and  procedures  that  you  have  to  deal  with, 
and  also  some  demonstration  of  a practicability  to 
apply  these  procedures  and  these  pieces  of  equipment  to 
this  particular  piece  of  hardware  which  you  have  to 
inspect . 
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We  thought  that  the  idea  of  using  protable  or  semi- 
portable test  samples  like  the  "Have  Cracks  Program" 
was  a good  one.  It  is  not  necessary  we  don't  think  to 
cart  these  things  around  the  country.  You  can  ship  them 
and  get  the  results  back  and  process  the  data  in  a 
central  point.  This  provides  a uniform  evaluation  of 
the  results  and,  I think,  simplifies  the  ' number  of 
pieces  that  you  need. 

The  question  was  asked  whether  certification  should  be 
central  or  dispersed.  We  felt  it  couldn't  really  be 
answered.  Centralized  certification  certainly  has  some 
advantages.  It  gives  you  uniformity  in  the  program.  It 
probably  allows  you  to  prepare  a more  detailed  program, 
perhaps  do  it  better.  But  we  felt  very  strongly  that 
any  central  certification  program  that  was  considered 
should  not  hold  up,  not  delay  the  implementation  of 
some  certification  program  at  every  place  where  NDI  is 
used . 


We  also  heard  a lot  of  things  about  the  organization  of 
the  NDI  effort.  And  we  thought  that  that  should  be 
addressed,  not  having  realized  that  so  many  other 
people  would  address  the  same  problem.  So,  we  have 
addressed  it.  We  felt  that  NDI  should  be  in  quality 
management . 

We  also  want  to  make  a point  strongly  that  --  out 
English  friends  made  the  point  very  effectively  — that 
one  of  the  prime  things  that  you  need  is  a recognition 
of  the  importance  of  NDI  because  it  is  a safety 
responsibil ity. 

This  is  essentially  the  story.  I guess  I added  one  line 
here  which  the  rest  of  my  team  doesn't  know  about  yet. 
I think  you  have  got  to  do  that. 

"Make  it  happen."  I heard  some  comments  and  discussions 
yesterday  that  indicated  a defeatist  attitude.  We  have 
got  all  this  hierarchy  above  us,  and  there  isn't  really 
much  we  can  do  about  it  and  so  on.  I think  that  if  you 
take  that  kind  of  attitude,  that  is  how  it  stays.  You 
have  to  go  to  and  make  it  happen.  That  is  all  I have  to 
say. 

Questions? 
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UNIDENTIFIED  SPEAKER: 


Was  it  the  opinion  of  your  group  that  the  Air  Force 
should  have  a certification  program? 

MR.  WHEELER;  Yes 

UNIDENTIFIED  SPEAKER:  Was  that  unanimous? 

MR.  WHEELER:  I asked  for  negative  opinions.  And  there 

were  none. 

UNIDENTIFIED  SPEAKER: 

Will  you  clarify  certification:  Is  it  to  the  existing 
requirements;  that  is,  SNT-TC-1A  which  includes 
practical  examination?  Or  is  it  something  that  we  have 
not  done  vet  like  the  demonstration  program  as  part  of 
the  certification? 

MR.  WHEELER: 

As  I understand  what  was  done  by  this  one  group,  they 
— the  question  was:  What  is  meant  by  certification  in 
this  case.  I talked  about  some  certified  people.  The 
question  is:  What  is  that. 

As  I understand  it,  these  were  a group  of  people  where 
selection  of  the  personnel  is  exercised  and  where  they 
are  given  --  let  me  back  off  --  where  they  have  initial 
training.  They  are  then  considered  nominally  certified. 
But  when  they  reach  the  base,  I will  say,  they  are 
treated  as  trainees.  And  they  are  not  considered  by  the 
base  to  be  certified  nor  do  they  do  any  NDI  on  their 
own  until  they  have  had  considerable  on-the-job 
training . 

That  on-the-job  training  requires  a formal  qualifica- 
tion examination.  And  they  don't  do  NDI  on  the  aircraft 
until  they  have  passed  those  qualification  examina- 
tions . 


Is  that  essentially  correct?  It  is  a local  policy  at 
one  location.  Is  that  essentially  correct? 


UNIDENTIFIED  SPEAKER: 

The  only  thing  that  is  different  is  that  the  training 
was  on  the  base. 

UNIDENTIFIED  SPEAKER: 

Is  there  any  thought  in  recognition  of  the  certifica- 
tion both  in  military  and  in  civilian  production?  Would 
the  same  certification  be  recognized  by  the  military  as 
well  as  civilian  production? 

MR.  WHEELER: 

The  question  was:  Will  these  certifications  be  the  same 
for  both  military  and  civilian  and  will  they  be 
accepted  by  the  other  organizations.  Is  that  right? 

UNIDENTIFIED  SPEAKER:  Right. 

MR.  WHEELER: 

We  did  not  address  that  particular  problem.  But  it 
would  seem  to  me,  speaking  as  an  individual,  that  if 
they  are  suitable  to  the  task,  they  ought  to  be  the 
same . 

UNIDENTIFIED  SPEAKER: 

May  I speak  for  the  industry,  please  --  the  utilities? 

From  the  utilities  standpoint  — I speak  of  Georgia 
Power  Company  — we  recognize  all  training  regardless 
of  whether  it  is  obtained  in  the  military  or  not.  As  a 
matter  of  fact,  we  like  to  have  military.  However,  just 
by  virtue  of  the  training  they  are  not  automatically 
placed  in  the  job.  They  are  administered  various  quali- 
fication tests  because  we  adhere  to  the  SNT-TC-1A. 

MR.  CAUSTIN: 

This  is  a comment,  not  a question.  The  other  day  during 
the  presentation  they  were  talking  about  — one  of  the 
questions  came  up  --  what  do  you  do  with  these  people 
that  are  low  performers,  what  airplane  do  you  allow 
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them  to  work  on.  Certification  gives  you  folks  the 
opportunity  to  eliminate  those  low  performers  from  the 
NDI  circuit.  And  that  is  the  main  reason  for 
certification . 

In  my  point  of  view  it  allows  you  to  get  rid  of  the  low 
performers . 

MR.  WHEELER: 

We  completely  agree. 

MR.  DORGAN: 

This  is  more  of  a statement  than  a question.  Being  in 
the  military,  we  are  going  to  be  up  against  somewhat  of 
a problem  and  we  are  opening  up  some  type  of  a 
Pandora's  box  when  we  keep  training  these  people,  we 
keep  certifying  these  people.  We  are  going  to  certify 
ourselves  out  of  business  because  as  soon  as  industry 
realizes  we  have  got  these  highly  trained  and  certified 
people  within  the  military  where  do  you  think  they  are 
going  to  go  to  earn  the  money. 

So,  these  are  some  of  the  things  we  are  going  to  have 
to  recognize  and  kind  of  reward  these  people  for  the 
education  and  certification  they  are  going  to  get. 

MR.  CAUSTIN: 

You  are  talking  about  where  they  go.  I think  the 
industry  would  like  to  have  the  Air  Force  to  have 
certified  people.  I think  it  would  lessen  our  problems. 

UNIDENTIFIED  SPEAKER: 

I have  a question.  George,  you  mentioned  that  in  the 
top  25  percentile  37  percent  of  the  samples  that  were 
taken  were  certified  individuals.  Do  you  have  any  feel 
for  the  number  of  certified  people  in  the  lower  10 
percent  bracket? 
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MR.  WHEELER: 


We  looked  at  that  only  for  one  of  the  examinations.  And 
I don't  recall  now  which  one  it  was  --  But  it  was 
Sample  F on  sonic  test.  And  that  was  one  in  which  no 
certified  people  appeared  in  the  top  25  percent.  So,  we 
looked  at  the  bottom  10.  And  there  were  three  of  them 
in  the  bottom  10  on  that  one  examination.  There  are 
some  anomalies  in  the  data  as  I said,  and  that  is  one 
of  them. 

UNIDENTIFIED  SPEAKER: 

The  reason  why  I asked  that  was  in  reference  to  Ed 
Caustin's  statement  about  implementing  a certification 
program  and  therefore  you  would  eliminate  low  per- 
formers. I don’t  agree.  I only  feel  that  this  is  going 
to  happen  when  the  certification  program  is  correctly 
applied.  And  if  we  don't  have  a certification  program 
that  is  proper  and  if  it  is  not  implemented  properly, 
it  is  not  going  to  eliminate  low  performers. 

MR.  WHEELER: 

Well,  if  you  don't  do  it  right,  it  is  not  going  to 
accomplish  anything.  But  I think  that  in  general  the 
application  of  a certification  program  which  usually 
means  some  kind  of  test  administered  at  the  end,  tends 
to  weed  out  the  poor  performers.  A lot  of  them  just 
won't  play.  They  won't  be  interested  if  they  know  they 
are  going  to  be  examined  at  the  end. 

Sam  is  pointing  out  that  the  general  level  of  per- 
formance on  this  Sample  F on  sonic  testing  was  poor. 
The  mean  detection  for  a half-inch  radial  flaw  was 
about  15  percent. 

There  were  a couple  others  back  there.  Don. 

UNIDENTIFIED  SPEAKER: 

This  is  a comment  in  that  all  certification  programs 
that  I have  read  or  to  hold  the  documents  indicate  that 
the  certification  implies  --  the  word  implies  responsi- 
bility of  the  employer.  In  other  words,  he  accepts  the 
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responsibility  of  doing  this  certification.  The  ques- 
tion is  who  is  the  employer  in  the  Air  Force  or  who  is 
going  to  accept  this  responsibility.  Is  it  the  depot, 
or  is  it  the  Air  Force  in  general? 

MR.  WHEELER: 

I guess  my  answer  to  that  is  --  it  is  not  something  we 
dealt  with  yet  — it  depends  on  how  you  set  it  up.  If 
you  set  up  a central  certification  program  in  the  Air 
Force,  it  is  going  to  be  one  thing.  What  we  are 
advocating  is  you  should  do  something  at  every  depot. 
Do  not  use  unqualified  people.  It  is  easy  to  be  in 
favor  of,  I realize,  and  harder  to  implement.  But  if 
you  don't  try  you  are  never  going  to  get  there. 

UNIDENTIFIED  SPEAKER: 

You  made  the  statement,  down  where  you  added  on,  "Make 
it  Happen".  You  are  one  who  has  been  pretty  successful, 
in  fact  would  you  give  us  some  suggestions  on  how  to 
make  it  happen  without  losing  your  30b. 

MR.  WHEELER: 

The  question  was  to  the  effect  that  I have  been  able  to 
make  it  happen  myself  and  how  should  you  make  it 
happen.  I will  see  you  later. 

Obviously,  there  is  no  one  way  to  do  it.  As  it  happens, 
I don't  have  to  have  anyone  certified  to  anyone  else's 
requirements.  I certify  my  people  because  I think  they 
ought  to  be.  And  I find  it  useful. 

Yes,  sir. 

UNIDENTIFIED  SPEAKER: 

I would  like  to  go  back  to  the  previous  question.  In 
the  Air  Force  we  have  to  look  to  our  program  manager 
for  NDI  to  answer  that  question  of  who  is  going  to 
initiate  certification  in  the  Air  Force. 
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MR.  WHEELER: 


His  question  is  who  is  going  to  initiate  the  certifi- 
cation in  the  Air  Force. 

MR.  BOISVERT: 

I have  been  told  we  will  have  a certification  program. 
It  is  a matter  of  working  out  the  details  and  so  on. 
How  we  are  going  to  do  it,  who  is  going  to  administer 
it,  these  are  details  we  still  have  to  be  taking  under 
advisement.  We  are  still  arguing  between  Tom  Cooper 
sitting  there  holding  his  head  and  shaking  yes  and  no. 
But  there  will  be  a program.  There  will  be  a program 
of,  rather  than  certification,  we  are  talking  about 
identifying  technician  proficiency. 

If  I could  identify  the  marginal  performers,  if  I can 
identify  those  performers  that  need  additional  training 
or  additional  upgrading  in  capabilty,  and  if  I can 
identify  my  high  performers,  people  that  I can  trust  on 
fracture  critical  parts,  a green  card  inspector  if  you 
want,  then  we  will  be  one  step  forward. 

MR.  WHEELER: 

Let  me  come  back  a minute  to  the  question  Mickey  asked. 
I didn't  mean  to  slough  it  off  particularly.  I think 
there  are  a lot  of  different  answers  to  different 
situations . 

The  thing  I would  like  to  say,  though,  is  that  the  kind 
of  data  that  is  available  here  — although,  it  is 
pretty  sparse  in  this  case  --  if  you  can  find  more  data 
like  that  — and  I think  you  can,  perhaps  not  on  such  a 
broad  basis,  but  I think  you  can  find  evidence  that 
certification  does  indeed  help.  And  if  you  cart  that 
around  through  your  management,  particularly  if  you 
have  a little  trouble  sometime  and  someone  gets  hurt, 
which  I would  not  advocate  as  a way  to  have  it  happen, 
but  that  helps.  That  really  helps. 

Anything  else? 

Thank  you. 
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EQUIPMENT 


TASK  GROUP  LEADERS: 

Ron  Seiner 
D.  L.  Conn 
R.  M.  Neufeld 

Mr.  Ron  Seiner,  Summary  on  Equipment:  I gave  a lot  of 
thought  about  how  to  summarize  this.  I thought  about 
going  over  the  questions  one  by  one  making  specific 
comments  to  the  particular  questions.  I sat  and 
listened  to  the  tapes  last  night  for  about  two  hours 
and  decided  that  was  an  impossible  task  to  get  too  many 
summary-type  answers  that  a lot  of  people  agreed  on. 
So,  after  sitting  and  listening  to  the  tapes  for  a long 
period  of  time  I read  through  the  notes  and  began 
looking  for  similar  comments  on  certain  issues,  certain 
parts  of  the  equipment  program.  And  not  seeing  too  much 
continuity  there  I went  to  bed,  got  up  rather  early 
this  morning  and  sat  and  mulled  over  it  and  said  I 
think  Bernie  would  appreciate  it  if  I winged  it. 

Really  what  I did  was  I came  up  with  mostly  general 
comments,  general  suggestions  for  how  our  group  felt  or 
the  three  groups  that  we  had  in  our  session  on  equip- 
ment felt  about  the  equipment  area  in  the  NDI  relia- 
bility program.  So,  I am  going  to  show  you  some  general 
comments  first.  In  the  latter  part  we  will  have  some 
specific  comments. 

Now,  we  weren't  originally  assigned  the  task  of  talking 
about  standards,  in  fact,  we  tried  to  avoid  it.  We  did 
the  first  two  sessions.  We  got  started  on  the  third 
session  and  some  gentleman  said,  "I  came  in  here 
strictly  to  talk  about  standards."  So,  we  opened  it  up. 
We  kicked  standards  around  for  about  30  minutes.  So,  we 
will  have  a couple  of  comments  to  make  about  standards. 

I think  what  I would  like  to  do  is  just  go  through 
without  a great  deal  of  detail  and  open  it  up  to  the 
floor  to  see  what  the  audience  thinks  about  it  later 
on.  There  was  much  comment  about  automated  equipment, 
how  much,  when,  where,  and  how.  How  much  intelligence 
should  be  built  into  that  system?  Should  we  take  away 
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the  complete  control  of  the  operator  and  make  it  pos- 
sible to  press  a button  to  center  the  equipment,  number 
two,  press  another  button  and  complete  a scan,  and 
number  three,  press  another  button  and  accept  or  reject 
with  some  built-in  intelligence  in  the  system. 

Once  we  got  past  an  automated  system  there  wasn't  that 
much  general  agreement.  But,  basically,  there  was 
considerable  agreement  in  that  we  need  more  automated 
systems  when  practical,  of  course,  and  when  econom- 
ically feasible. 

We  certainly  cannot  devise  an  automated  system  for 
every  testing  situation  that  occurs  or  is  spelled  out 
in  the  -36.  But  there  are  many  high  volume-type 
inspections  that  lend  themselves  very  well  to  automated 
systems  and  certainly  not  only  would  improve,  speed  up 
the  results,  that  would  improve  the  reliability. 

When  we  get  to  the  topic  of  combining  the  search  and 
the  flaw  characterization  operation,  again  it  seems  to 
follow  quite  naturally  that  is  we  have  a high  volume 
inspection  system  — now,  we  are  talking  about  in- 
service  inspection.  There  was  some  disagreement  to 
having  a combined  search  and  flaw  characterization 
system  as  far  as  industry  was  concerned.  But  as  far  as 
the  Air  Force  for  in-service  inspection  it  would  be 
very  helpful  if  we  could  combine  the  search  and  the 
flaw  characterization  operations  on  our  high  volume 
automated  systems  applications.  That  is  number  two. 

Okay,  now  we  get  into  a slightly  different  topic.  We 
talked  a lot  about  having  more  automated  systems,  of 
course,  which  add  to  the  cost.  We  talked  about  adding 
and  implementing  flaw  characterization  into  the 
operation  to  make  it  more  intelligent,  to  hopefully 
increase  our  reliability.  This  is  great  if  we  can 
afford  the  system  in  the  beginning.  But  we  have  also 
got  to  realize  that  by  adding,  more  complexity  to  the 
system  and  more  extensive  systems  in  the  beginning,  we 
are  going  to  add  to  already  existing  problems  of  repair 
and  life  policies.  In  other  words,  when  the  equipment 
gets  10-15  years  old,  it  craps  out  and  we  don't  have 
anything  programmed  to  replace  it.  So,  it  is  felt  like 
if  we  do  implement  more  automated  systems  and  more 
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complex  intelligence  systems,  we  better  have  more 
uniform  repair  and  lifing  policies.  By  this  I mean  that 
if  we  have  an  X-ray  system  that  breaks  down,  it  would 
be  very  helpful  if  we  had  a very  uniform  policy  on  what 
we  do  to  repair  this  system.  Some  bases  send  it  to 
PM  EL , some  bases  get  it  repaired  locally  at  an  elec- 
tronic repair  outfit,  some  send  it  back  to  the  manu- 
facturer. Right  now  there  is  not  any  really  clearcut 
repair  and  lifing  policies  that  we  could  see  in  the  Air 
Force.  So,  it  would  certainly  help  if  we  are  going  to 
more  expensive  and  complex  systems  if  we  institute  some 
kind  of  repair  and  lifing  policies. 

It  was  also  felt  that  if  we  add  more  complex  systems, 
more  automation  to  the  inspection  process,  we  better 
make  darn  sure  that  we  provide  standards.  This  is 
particularly  a 'must'  situation  for  highly  automated, 
complex  systems. 

It  was  also  felt  that  in  addition  to  this  in  many 
instances  in  practice  today  in  the  -36  a particular 
inspection  procedure  spells  out  a standard.  Now,  the 
question  is  where  do  you  get  the  standard  and  how  do 
you  reproduce  it.  There  is  no  one  place  where  we  can  go 
to  get  that  standard.  So,  if  we  want  to  have  a standard 
for  a particular  inspection  set  up,  we  have  to  make  it 
or  have  it  prepared  locally  or  find  somebody  that  might 
have  one  that  is  similar  to  the  part  that  we  are  in- 
specting. So,  it  is  felt  that  the  manufacture  of 
standards,  particularly  in  high  volume  inspection  areas 
and  in  other  areas  where  they  would  be  extremely  useful 
in  increasing  your  reliability  for  example  in  the 
introduction  of  new  type  components  — here  I am 
referring  to  composite  skin  materials,  composite  skin 
not  only  over  skins  but  also  over  honeycomb  areas.  It 
is  almost  a must  that  we  have  a standard.  And  where  do 
we  get  it?  It  was  suggested  that  the  logical  way  to  do 
this  might  be  to  write  in  to  the  basic  procurement 
document,  when  you  buy  the  system,  you  also  buy  some 
standards.  Just  like  we  are  also  buying  the  -36  's  now. 
We  need  to  implement  a better  reference  standard 
system. 

The  next  comment  kind  of  goes  along  in  line  with  many 
factors.  But  it  is  felt  like  if  we  introduce  a lot  of 
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complex,  highly  automated  systems,  we  are  also  going  to 
increase  the  amount  of  boredom  on  the  part  of  the 
operator.  We  have  got  to  keep  this  guy  alive.  We  don't 
want  him  to  just  go  out  there  and  press  buttons. 

So,  there  was  a general  concensus  that  it  would  be 
highly  desirable  in  many  cases,  again  the  situation  has 
to  dictate  it,  but  wherever  possible  and  practical  that 
permanent  records  such  as  strip  chart  reportings,  C 
scans,  maybe  just  a sketch  of  the  part  being  inspected 
with  the  location  of  the  defects  plotted,  some  kind  of 
a visual  display  picture,  something  that  you  can  go 
back  to.  This  is  highly  desirable  when  you  are  working 
on  new  components  or  areas  where  you  do  not  have  a very 
good  expertise,  and  you  are  trying  to  get  your  data.  We 
need  baseline  data.  We  also  need  to  keep  the  operator 
vigilant.  We  feel  like  some  kind  of  permanent  records 
wherever  practical  and  feasible  are  needed.  Along  with 
keeping  the  operator  vigilant,  we  also  felt  like  in 
many  cases  to  assist  the  operator  out  in  the  field  that 
some  kind  of  a visual  display  would  be  most  helpful. 
This  could  take  the  form  in  most  cases  of  a cathode  ray 
tube  presentation.  Whenever  ultrasonics  is  being  used, 
a meter,  the  use  of  eddy  currents,  some  kind  of  a 
light.  Something  has  got  to  keep  this  operator 
vigilant.  We  just  cannot  let  him  go  along  and  press  a 
button . 

If  he  doesn't  have  any  real  way  of  knowing  that  the 
test  is  operating,  we  are  going  to  put  him  in  trouble 
and  put  him  to  sleep.  So,  along  the  lines  of  keeping 
the  operator  vigilant  and  not  making  his  job  one  of 
boredom,  we  felt  it  was  desirable  to  have  some  kind  of 
a visual  display. 

And  those  items  five  and  six,  the  permanent  records  and 
the  visual  displays  we  felt  like  were  particularly 
desirable  in  the  automated  systems  for  operator 
vigilance.  And  the  latter  case,  of  course,  is:  The 
permanent  records  are  desirable  when  you  are  trying  to 
establish  some  kind  of  a base  line. 

One  other  item  that  kept  coming  up,  which  is  kind  of 
directly  opposite  to  the  problem  of  making  our  equip- 
ment more  automated,  more  complex,  was  a lot  of  comment 
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about  simplicity.  We  want  to  make  this  thing  do  more, 
build  in  more  reliability,  have  it  to  have  more 
intelligence;  but  for  God's  sake  keep  it  simple.  Well, 
it  is  kind  of  hard  to  keep  something  simple  if  you  keep 
adding  more  to  it,  keep  adding  more  functions.  But  it 
is  felt  that  for  many  of  the  routine  inspections  that 
some  kind  of  programmable  material  is  needed  to 
increase  the  reliability.  And  this  would  also  make  it 
possible  for  you  to  go  and  do  the  equipment  setup  in 

some  kind  of  a reproducible  and  some  kind  of  a highly 

reliable  manner. 

Some  of  the  things  that  were  kicked  around  and  some  of 
the  general  ways  of  programming  this  so  we  could  go 

back  and  set  up  our  equipment  to  do  the  test  would  be 

in  the  form  of  a card  or  a chip.  Now,  we  are  talking 
here  primarily  about  the  electronic  controls,  where  do 
we  set  the  gain,  what  frequency  do  we  put  it  on,  how  do 
we  take  that  data,  what  do  we  do  with  that  data  after 
we  get  it.  So,  there  was  some  talk  about  cards  and 
chips.  Now,  of  course,  programming  is  another  problem. 
When  you  start  going  into  programming  you  get  into  the 
complexity.  But  there  are  some  situations  where  you 
have  high  volume  inspections  where  it  would  be  feasible 
and  practical  to  introduce  some  kind  of  programming. 

Those  are  the  basic  summaries  that  we  came  up  with.  I 
think  at  this  time  we  would  like  to  open  it  up  to  the 
floor  for  some  general  comments. 

Would  anybody  like  to  make  a comment? 

Yes,  sir. 

UNIDENTIFIED  SPEAKER: 

On  the  uniform  repair  policies  — 

MR.  SELNER: 


What  the  gentleman  said  was:  If  they  don't  change  the 
equipment  for  the  next  10  years,  they  would  like  to  see 
some  form  of  a uniform  repair  policy. 
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UNIDENTIFIED  SPEAKER: 

I didn't  sit  in  on  any  of  your  sessions  because  we  were 
busy  with  our  own,  but  one  of  the  things  that  was  not 
mentioned  in  your  talk  or  didn't  seem  to  be  discussed 
was  it  appears  to  me  that  like  for  certain  eddy  current 
inspections  or  ultrasonic  inspections  you  need  specific 
or  odd-shaped  probes  and  holders. 

Now,  does  the  manufacturer  supply  those  with  the  T.O., 
or  who  fabricates  these  special  devices? 

MR.  SELNER: 

Yes.  This  was  brought  up  along  with  the  question  of 
standards.  Generally  speaking,  probes  and  aids  for 
accomplishing  the  inspection  like  various  angles  or 
manipulators  or  guides,  templates  and  the  like  can  be 
obtained  fairly  easily  from  the  depots.  That  is  not 
that  big  of  a problem.  That  is  my  feeling  and  what  the 
general  feeling  was.  This  was  discussed.  This  might  be 
a problem  that  I am  not  aware  of  and  some  of  the  other 
people  were  aware  of.  It  is  not  as  big  of  a problem  for 
example  as  that  of  standards. 

UNIDENTIFIED  SPEAKER: 

May  I make  a comment  — Pratt  and  Whitney  Aircraft.  You 
must  have  Grand  Master,  especially  on  engines.  The 
Grand  Master  and  all  of  the  reference  masters  can  be 
bumped  against  that  Grand  Master  so  all  the  bases  have 
exactly  the  same  thing.  So  in  the  event  one  gets  lost 
or  damaged  or  destroyed,  you  can  go  back  to  the  Grand 
master  and  everybody  is  conforming  exactly  to  the  same 
reference  masters,  the  same  probes  — especially  in 
eddy  currents. 


MR.  SELNER: 

Your  question  is  kind  of  --  your  statement,  right.  You 
are  talking  about  a Grand  Master  as  far  as  equipment 
goes , probes? 
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UNIDENTIFIED  SPEAKER: 


The  manufacturer,  vendor,  usually  makes  reference 
masters.  There  must  be  a Grand  Master  to  this  manu- 
facturer. And  all  of  the  other  masters  that  are  sent 
out  to  the  field  must  be  checked  and  calibrated  and 
correlated  with  that  and  sent  to  the  field.  In  the 
event  one  of  them  gets  lost  or  damaged  or  something, 
another  one  can  be  manufactured  at  the  manuf actur er ; 
i.e.,  Pratt  and  Whitney,  General  Electric  or  whatever. 
We  have  experienced  this  quite  a bit.  The  military 
seems  to  have  a tendency  to  lose  things  and  damage 
things.  There  are  many  times  that  we  have  to  have  them 
make  duplicate  orders.  And  they  have  to  have  something 
to  go  back  to.  You  can't  go  out  to  some  vendor  and  have 
them  make  something  which  is  similar  to  it. 

MR.  SELNER: 

That  is  a good  comment. 

Any  other  comments  or  questions? 

UNIDENTIFIED  SPEAKER: 

I am  just  goig  to  add  that  one  of  the  questions  that 
came  up  repeatedly  with  that  one  outfit  wants  a very 
complicated  instrument  with  a microprocessor  in  it.  The 
next  one  wants  a simple  one  with  an  on-and-off  switch. 
One  of  the  comments  we  got  repeatedly  from  the  vendors, 
equipment  manufacturers  is  that  they  can't  really 
afford  to  develop  just  a few  instruments.  This  is  a 
question  that  was  thrown  out:  Who  is  going  to  develop 
these  new  instruments,  who  puts  the  money  in  to  develop 
them? 

MR.  SELNER: 

Just  like  in  everything,  if  you  get  one  thing,  you  lose 
something  else.  So,  you  have  to  trade  things  off. 

Yes,  Don. 
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UNIDENTIFIED  SPEAKER: 


In  preparing  the  T.O.'s  or  the  procedures,  I might  be 
wrong  but  it  appears  to  me  that  the  government’s 
statement  says  that  you  have  to  use  equipment  that  is 
in  the  inventory.  Now,  how  does  the  Air  Force  get  other 
equipment  which  is,  say,  on  the  market  which  may  be 
better  than  the  equipment  which  they  are  calling  for  to 
do  the  job? 

MR.  SELNER: 

Do  you  want  to  tackle  that  one,  Bernie? 

MR.  BOISVERT: 

Well,  first  of  all  we  are  limited  to  our  budget.  It 
would  be  lovely  if  I had  unlimited  bucks.  I could  do  a 
lot  of  great  things.  We  just  don't  get  unlimited  bucks. 
The  spec  really  reads  - it  says:  Hey,  we  have  existing 
equipment.  We  have  ultrasonic  equipment,  eddy  current 
equipment.  X-ray  equipment,  so  on.  If  we  have  brand  "X” 
and  you  feel  brand  "Y " is  necessary  and  you  can  prove 
it  to  me  so  I can  go  satisfy  the  bean  counter  and  get 
some  bucks  to  buy  brand  "Y",  then  there  is  that  option. 
But  we,  I,  and  I believe  everyone  else  in  here  have 
certain  preferences.  And  I like  brand  "X".  When  I pick 
up  the  phone  and  call  and  say,  Charlie,  I have  got 
trouble  with  my  unit,  he  is  there  johnny  on  the  spot, 
fixes  my  unit,  and  I am  back  in  business. 

I tend  to  prefer  him.  But  that  is  not  the  way  we  can 
afford  to  buy  equipment.  We  have  to  buy  the  low  bid 
equipment  that  does  this  performance  criteria.  Right 
now  somebody  has  come  in  and  shown  us  — they  haven't 
put  it  in  writing  yet  and  that  is  the  one  thing  I have 
got  to  get  on  you  all  about.  Put  it  in  writing.  They 
have  shown  us  where  one  piece  of  equipment  is  necessary 
for  a specific  job  on  one  of  our  newer  systems.  It  is 
going  into  the  TA.  And  we  are  going  to  have  that,  and 
everybody  else  can  use  it,  too.  We  do  encourage  this. 
As  new  pieces  of  equipment  are  developed,  we  would  like 
to  see  if  you  can  show  us  where  this  would  do  the  job 
and  the  other  one  won't.  But  I can't  drive  a Cadillac 
when  a Ford  automobile  will  do. 


217 


That  is  really  my  guidelines  in  buying  equipment. 
UNIDENTIFIED  SPEAKER: 

Did  your  task  group  use  any  of  the  computer  data  to  see 
if  equipment  did  make  any  difference  on  the  probability 
of  detection? 

UNIDENTIFIED  SPEAKER: 

No,  we  didn't  because  not  enough  information  was 
reported.  Very  general,  transducer  and  serial  number, 
for  example.  That  is  one  thing  that  we  would  like  to 
suggest  next  time,  that  if  this  experiment  is  repeated 
that  much  more  definitive  information  be  stored  in  the 
computer  to  allow  this  kind  of  analysis. 

MR.  SELNER: 

Did  you  hear  all  that  in  the  back? 

Ward,  you  had  a question. 

UNIDENTIFIED  SPEAKER: 

I have  a comment.  One  way  of  working  equipment  problems 
for  unique  applications  is  to  identify  it  as  a tool  and 
supply  it  as  a tool.  And  we  will  all  use  that. 

MR.  SELNER: 

Ward's  comment  was  that  part  of  the  solution  to  the 
problem  of  ultrasonic  equipment  is  to  identify  it  as  a 
tool.  I am  not  quite  sure  what  the  implication  is 
there . 

Are  there  any  other  comments  or  questions? 

Yes,  sir. 

UNIDENTIFIED  SPEAKER: 

Could  we  initiate  into  this  program  some  means  of 
getting  standardization  among  equipment  such  as  the  AC 
cord  to  all  equipment?  Each  one  of  them  comes  out  with 
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a separate  type  of  plug.  So,  in  our  inventory  we  must 
stock  at  least  15  different  types  of  AC  cords  just  to 
fit  individual  units  when  one  back-up  cord  for 
maintenance  would  be  sufficient. 

MR.  SELNER: 

This  is  a comment  that  can  open  up  a real  can  of  worms. 
I left  this  out  for  a reason.  As  you  all  know  this  is 
one  of  the  topics  that  keeps  getting  kicked  around,  the 
one  of  standardization;  not  only  equipment,  for 
example,  connectors,  output  probes,  power  cords,  it 
includes  nomenclature  as  well.  I am  sure  that  we  would 
all  agree  --  I don't  think  we  get  any  real  dissenters 
other  than  probably  the  equipment  manufacturers  — it 


would  certainly  be  advantageous 


both 


industry  and  government,  if  we  had  standardization  both 
of  equipment  and  nomenclature.  And  it  is  fairly  obvious 


that 

that 


savings 
have  to 


not  only  in  time  but  the  materials 
store  in  your  laboratory,  all  the 


adapters  for  all  these  eddy  current  probes  and  differ- 


ent types  of  connectors 
creates  problems. 


the  ultrasonic  probes,  it 


It  would  be  nice  if  we  could  get  the  equipment  manu- 
facturers to  make  their  things  fairly  interchangeable 
and  agree  on  terms. 


UNIDENTIFIED  SPEAKER: 

I have  not  been  going  to  ASTM  meetings  lately,  but 
isn't  there  an  ASTM  committee  on  standari zation?  Does 
anyone  in  the  room  know? 

UNIDENTIFIED  SPEAKER: 

Yes,  Supposedly  there  will  be  an  ASTM  specification 
combining  nomenclature  and  connectors  for  NDT  equip- 
ment. Is  that  correct? 

UNIDENTIFIED  SPEAKER: 


Would  you  repeat  the  answer? 


UNIDENTIFIED  SPEAKER: 


What  I said  was  that  the  committee  that  he  is  referring 
to,  the  ASTM  committee  on  nondestructive  testing,  this 
committee  has  produced  the  documents  on  the  standard- 
ized nomenclature.  But  as  far  as  I know  there  is 
nothing  on  standardized  power  cords,  connectors,  equip- 
ment, so  on. 


MR.  SELNER: 


Bill  Scruggs,  you  have  a comment? 

MR.  SCRUGGS: 

This  is  going  back  to  Don's  comment  on  the  data  that  we 
have  in  the  computer.  There  is  a modest  amount  of 
information  concerning  equipment  performance  from  all 
bases.  We  did  check  for  instance  linearity  of  both  the 
sweep  and  vertical  deflection  on  ultrasonic  equipment. 
We  ran  step  wedge  shots  and  test  developer  strips  on 
X-ray  equipment  and  so  forth.  So,  we  do  have  this  kind 
of  information.  It  is  semi-quantitative.  We  have  serial 
numbers  on  the  pieces  of  equipment  used  in  all  the 
tests  that  were  performed.  So,  there  is  a fair  amount 
of  documentation  in  that  area  stored  in  the  computer 
and  will  be  available  to,  for  instance,  look  at  results 
compared  to  equipment  performance. 

MR.  SELNER: 


MR.  BOISVERT: 

In  any  complex  organization  --  and  you  can  compare  the 
Air  Force  to  General  Electric  --  we  have  got  196  bases 
that  need  equipment  throughout  the  world.  Navy  has  a 
similar  amount.  So  we  are  a big  operation.  Let's  face 
it.  We  have  got  20  major  commands.  Each  has  a little 
different  mission,  a little  different  application.  So, 
recognize  this  as  a problem.  To  answer  the  specific 
question,  quality  control  of  incoming  material,  we 
normally  try  to  check  the  first  model  of  a procurement 
buy.  If  we  buy  ultrasonic  units,  we  check  the  first 
model  of  this . 

We  then  depend  upon  the  DCAS  people,  defense  contracts, 
administration,  DCASR , DCASA  people  to  assure  that  the 
rest  of  the  models  are  the  same  way.  When  we  get  them 
at  Kelly,  no,  we  don't  check  them.  We  don't  even  open 
the  box.  Normally,  we  send  them  out.  In  fact,  many 
times  we  ask  the  manufacturer  to  deliver  them  to 
Wr ight-Pa tterson , or  whatever  Air  Force  Base. 

MR  CAUSTIN: 

Do  you,  Kelly  Air  Force  Base,  give  DCAS  the  test  accep- 
tance instructions  that  you  want  them  to  do,  or  is  that 
left  up  to  them? 

MR.  BOISVERT: 


No . We 

ask 

the  manufacturer 

to 

submit 

a proposed 

accetance  test  procedi 

ure.  We 

ask 

him  to 

submit  to 

f i rst  , 

an  art 

:icle  test 

where  we 

go 

through 

and  do  all 

of  the 

good 

things  . 

Then  the 

acceptance 

test  pro- 

cedures,  and  this  we  provide  to  the  DCAS  people. 

Relative  to  Mil.  Specs.,  great  documents.  Unfortun- 
ately, a Mil.  Spec,  requires  a design  engineer.  And  you 
equipment  manufacturers  have  got  staffs  of  10-15  design 
engineers.  I am  sitting  at  Kelly  with  two  people.  We 
just  cannot  get  into  the  design  engineering  phase.  Let 
them  do  the  design,  and  we  will  just  buy  it.  That 
design  they  make  so  that  Mr.  Caustin,  Mr.  Hagemaier,  so 
that  everybody  is  using  the  same  equipment.  I am  trying 
to  use  the  same. 
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We  are  trying  to  use  the  same  equipment  that  the  air 
frame  manufacturers,  the  engine  manufacturers  are 
using . 

MR.  SELNER: 

I think  we  will  close  it  there. 


VI 


RELIABILITY  MEASUREMENT/HODELING 


TASK  GROUP  LEADERS: 

D.  J.  Hagemaier 
S.  Klima 
W.  D.  Rummel 


Mr.  W.D.  Rummel,  Summary  on  Reliability  Measurement  and 
Modeling:  In  looking  at  the  task  of  reliability  model- 
ing, we  decided  very  early  in  the  program  that  the  best 
approach  would  be  to  determine  that  top  10  percent, 
select  those,  clone  them,  and  get  rid  of  all  the  other 
guys.  That  eliminates  the  problem.  Now,  in  falling  back 
on  what  can  we  do  in  the  mean  time,  we  took  on  some 
objectives.  And  the  objectives  were  to  define  some 
approaches  to  modeling.  There  were  two  tasks  in  that, 
to  try  to  determine  what  the  critical  characteristics 
were  or  are  and  also  what  the  boundary  conditions  and 
assumptions  are  that  we  have  to  make  in  order  to  go 
forward  with  such  an  item.  Now,  in  starting  on  that  we 
looked  at,  first,  what  has  been  done  in  the  past  and 
what  should  we  learn  from  that. 


First  of  all,  in  the  past  programs  we 
look  at  the  capability  of  a technique 
output,  normally  in  terms  of  flaw  size, 
all  other  variations,  but  specifically 
minimize  the  human  factors. 


have  tried  to 
to  give  us  an 
by  minimizing 
attempting  to 


Now,  the  area  in  which  "Have  Cracks"  has  changed  that 
is  we  have  variable  human  factors  and  human  variables 
on  the  processes  so  that  we  have  two  things  coming 
together  which  are  changing  what  our  output  should  be. 
So,  we  have  a wider  spread  in  the  population.  But  we 
have  emphasized  in  this  case  the  human  factors  aspect. 
Now,  if  we  go  forward  with  that,  what  assumptions  can 
we  make,  what  lessons  have  we  learned?  First  we  say 
that  the  POD  or  probability  of  detection  is  a con- 
tinuous function  as  a function  of  flaw  size. 

We  see  that  a large  crack  and  a slightly  smaller  crack 
as  we  home  in  on  some  threshold  level  at  which  the 
detection  is  going  to  decrease,  as  we  approach  that 
small  crack  size  or  drive  in  on  it  we  don't  expect  to 
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see  any  dips  or  valleys  in  that.  So,  we  think  the  range 
is  not  important  in  terms  of  our  sampling.  We  simply 
see  that  as  a continuous  function.  And  that  is  very 
important  as  we  go  down  the  line. 

The  next  thing  is  we  assume  a representative  sampling 
of  the  population.  There  are  things  that  we  do  in  the 
statistical  design  of  the  experiment  to  assure  that, 
but  there  are  many  parameters  that  play  into  this.  So 
we  are  assuming  that  representative  sampling.  And  there 
are  some  areas  where  we  think  we  may  have  biased  that. 

The  third  item  is  that  we  assume  that  all  observations 
are  independent  of  each  other  so  that  one  man  looking 
at  a single  flaw  is  not  affected  by  the  next  flaw  that 
he  sees,  or  his  decision  is  not  affected  by  the  next 
flaw  down  the  line.  There  are  some  reasons  to  argue 
that,  but  that  has  to  be  one  of  our  assumptions. 

We  assume  a constant  calibration  for  whatever  calibra- 
tion will  mean.  And  for  purposes  of  this  program  and 
the  past  program  we  feel  it  is  necessary  to  assume 
constancy  of  method.  We  don't  at  this  point  see  a way 
to  transfer  X-ray  data  to  eddy  current  data  and  so  on. 

And  the  final  item  in  this  is  that  we  will  assume  that 
some  level  of  false  call-outs  or  false  reporting  is  a 
condition  of  performing  that  inspection.  Now,  that  is 
not  an  item  we  can  vote  on.  It  is  something  we  have  to 
accept  as  a condition  of  the  inspection.  Hopefully,  we 
choose  an  inspection  method  which  has  the  proper  leeway 
or  boundary  condition  so  that  that  value  is  very  low. 
But  that  is  one  of  the  things  we  do  have  to  live  with. 

Now,  if  we  go  and  look  for  common  denominators  in  all 
of  these  things  we  see  that  the  probability  of  detec- 
tion is  some  function  of  the  signal  to  noise  of  the 
operation.  That  includes  both  the  signal  to  noise  of 
the  inspection  output  or  all  the  process  steps  that  we 
are  going  through  and  the  manner  in  which  the  human 
responds  to  that  signal  level. 

Some  of  the  functional  elements  that  contribute  to 
that,  or  the  important  functional  elements  that 
contribute  to  that  are  the  method  selection,  the  human 
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factors  element,  the  equipment  and  standards  — and  we 
did  combine  those  --  and  the  test  object  or  the  item 
that  we  are  trying  to  interrogate.  Now,  in  terms  of 
method  selection  I think  we  were  not  redundant  with 
other  groups.  We  said  that  the  flaw  was  an  important 
swinger  in  type,  geometry,  size.  And  that  was  with  our 
length,  depth  and  width,  or  closure  being  part  of  the 
width,  location  and  orientation. 

And  the  test  object  itself  in  which  we  were  interrogat- 
ing to  find  the  flaw  was  important  in  material  shape, 
geometry,  access,  and  so  on.  The  environment  is  impor- 
tant to  selection  of  the  method. 

Now  these  factors  are  also  important  in  some  of  the 
other  areas  . 

If  we  go  to  the  human  factors  --  and  we  talked  a lot 
about  human  factors  --  we  see  the  two  primary  things  in 
being  the  human  response  to  the  stimuli  of  the  inspec- 
tion and  the  confidence  of  the  man  that  is  making  that 
decision  in  that  stimuli  or  his  decisiveness,  his 
responsibility,  and  his  risk  assessment.  And  that 
includes  both  risk  to  him  by  making  the  decision  and 
the  resultant  risk  if  he  makes  the  wrong  decision  to  a 
system  or  someone  else.  Now,  with  that  we  got  back  into 
experience.  We  think  experience  is  important.  We  think 
the  quality  of  that  experience  may  be  the  thing  that 
will  reduce  the  scatter.  We  think  that  is  extremely 
important . 

The  training  is  important,  but  the  quality  of  the 
training  is  important.  The  supervision  s important. 
Certainly  the  quality  of  the  supervision  is  important. 
His  motivation  — he  needs  a method  of  self-check  of 
his  work.  He  needs  a self-assessment  of  work  quality. 
He  needs  some  objective  peer  assessment  of  that  work 
quality  and  certainly  needs  an  objective  management 
assessment  of  that  work  quality. 

Those  factors  at  this  time  seem  to  be  totally  missing 
from  our  systems. 

Then  there  are,  of  course,  man/machine  interface 
equipment  problems  that  have  to  do  with  dexterity  and 
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so  on.  There  are  man/machine  interfaces  that  can  be 
worked  with  respect  to  the  output  stimuli  response.  One 
of  the  things  we  saw  is  in  going  to  automation  we 
changed  some  of  those  demands  on  the  man.  But  we  also 
greatly  improved  the  signal-to-noise  or  the  margin  that 
he  has  to  slice  to  make  the  right  decision. 

Finally,  his  environment  will  be  important  in  the 
decisions  that  he  makes. 

The  test  object  is  important  in  that  what  we  are 
looking  for  in  the  test  object  is  an  equivalent 
inspection  stimuli  to  translate  data  or  to  model  from 
one  situation  to  another.  We  think  that  that  can  be 
done  by  the  proper  approach  to  the  calibration  or  the 
calibration  and  setup,  taking  into  account  some  of  the 
things  that  we  talked  about  in  material  geometry  and  so 
on . 

An  important  item  comes  into  the  area  of  concentration. 
We  talked  a little  bit  yesterday  about  hot  spots.  If  a 
man  has  a task  that  he  thinks  is  attainable,  then  his 
approach  to  that  task  is  certainly  better.  So,  his 
ultimate  response  will  be  better.  We  think  that  may  be 
an  area  where  we  are  getting  benefit  in  looking  at  hot 
spot  areas. 

Equipment,  we  left  that  to  Group  IV.  We  didn't  try  to 
solve  all  problems  for  everyone. 

We  looked  at  the  questions  and  added  some  of  our  own. 
In  terms  of  the  hotness  of  the  issues  we  tried  to  list 
them  in  priority.  First  on  the  list  is  false  reporting. 
Everyone  is  interested  in  false  reports.  That  is  the 
highest  interest  issue.  The  majority  of  folks  thought 
it  was  not  just  merely  an  economic  factor  but  there  was 
some  other  information  intrinsic  to  false  reporting. 
But  no  one  was  able  to  put  rationale  or  an  approach  to 
how  we  should  handle  that  beyond  tracking  it  and 
reporting  it  at  this  time.  We  think  that  that  can  be 
handled  in  one  of  the  down  line  studies. 

< 

*ttie  majority  of  the  opinions  said  that  we  indeed  ought 
to  penalize  a man  for  false  reports  in  the  same  way 
that  college  entrance  exams  are  penalizing.  But,  once 


again,  no  rationale  for  that  other  than  that  is  some- 
thing that  has  been  done  and  is  recognized  as  one 
technique  in  trying  to  condition  the  human  response. 


The  second  issue  was  the  90/95  criteria  that  Tom  Cooper 
has  been  talking  about.  Is  it  necessary?  Our  majority 
opinion  says,  yes,  that  that  is  a necessary  condition, 
that  if  we  go  to  a lower  sampling  the  end  result  will 
be  that  we  must  make  a more  frequent  inspection  if  we 
want  to  maintain  the  same  confidence.  So,  we  are  not 
really  saving  money  by  backing  off  in  the  front  end. 

The  third  it^m  was:  Should  the  inspector  know  he  is 
being  tested.  We  had  a majority  "no"  opinion.  At  the 
same  time  that  majority  recognized  or  was  of  the 
opinion  that  probably  isn't  possible  in  any  of  the 
tests  that  we  do  or  any  of  the  testing  that  we  do  down 
line.  We  had  a minority  opinion  that  we  should  let  the 
man  know. 

And  the  rationale  behind  that  is  we  should  not  play 
games  with  the  man. 

In  terms  of  the  reliability  transfer  --  that  is  one  of 
the  questions,  is  it  a feasible  approach  --  we  think 
that  it  is.  And  the  model  that  we  are  proposing  here, 
or  this  approach  is  one  way  of  doing  it. 

A final  item  is:  Who  uses  the  data  and  can  get  rid  of 
some  of  the  information,  hone  it  to  a specific  user 
group?  At  this  point  everyone  is  interested  and 
everyone  wants  to  cut  it  a little  differently.  So,  it 
looks  like  we  need  to  record  all  of  the  boundary  con- 
ditions and  assumptions  along  with  all  of  the  quantita- 
tive information  that  we  take  at  this  time,  because  we 
don't  have  rationale  to  really  read  it  out. 

Finally,  we  took  on  the  challenge  of  windows.  We  see 
there  are  three  programs  or  three  actions  that  need  to 
be  taken. 

The  first  has  to  do  with  the  human  factors  area.  We 
found  this  is  probably  a convenient  approach  since  the 
Air  Force  also  has  organizations,  or  is  organized  along 
this  line.  We  need  to  study  the  man/machine  interfaces 
in  both  physical  and  mental  output  as  a function  of  the 
signa 1-to-noise  stimuli. 
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And  the  second  area  is  one  which  is 
put  into  perspective.  We  are  all 
signal-to-noise  voltmeter  response, 
of  that,  important  to  NDT,  is  the 


a little  harder  to 
familiar  with  the 
But  the  second  part 
pattern  recognition 


part.  So,  that  human  factor  study  needs  to  look  at  both 
the  signal-to-noise  from  the  standpoint  of  the  thing  we 
are  traditionally  looking  at  in  the  electronics  area 
plus  the  pattern  recognition  factor  that  goes  along 
with  it.  There  are  established  methods  that  the  human 
factors  folks  measure  that  can  address  themselves  to 
this  problem.  So  maybe  we  are  beginning  to  talk  the 
language  by  taking  this  approach. 

The  second  area,  in  the  equipment  level  we  think  we 
need  to  hone  in  and  emphasize  the  stimulus  that  the 
instrument  is  putting  out  in  both  level  --  and  that  has 
to  do  with  signal-to-noise  — and  the  pattern,  because 
both  of  those  items  fall  into  it.  So  as  we  drive  in  on 
equipment  improvements,  those  are  the  factors  we  are 
really  interested  in  improving  in  equipment.  All  the 
other  things  can  make  them  specialized  and  so  on  but 
they  don’t  do  this  job.  They  are  not  really  helping  us 
out  in  terms  of  working  a reliability  problem. 


think  we  need  to  validate  a modeling 
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Questions? 

UNIDENTIFIED  SPEAKER: 

I would  like  to  elaborate  a 
about  the  attention  factor 
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whatever  you  want 
and  know  they  are  being  tested.  I 
shown  repeatedly  in  almost  every 
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that  people  do 
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think  it  has  been 
psychological  test 
get  attention.  Just 
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recognize  their  activity  and  they  will  always  do 
better.  I think  this  result  is  somehow  in  the  "Have 
Crack-Will  Travel"  results.  And  we  should  look  at  the 
results  of  "Have  Cracks"  as  being  an  upper  limit.  It 
could  be  much  worse. 

MR.  RUMMEL: 

There  are  some  comments  that  we  thought  this  was  one 
area  in  which  "Have  Cracks"  did  a good  job  in  having  a 
long-term  series  of  inspections  so  that  whatever  group 

vigilance  would  occur  as  the  guy  got  used  to  the 

situation  should  have  been  worked  in  this.  We  are  not 
together  on  whether  that  is  a driving  factor  at  this 
point  or  not. 

UNIDENTIFIED  SPEAKER: 

Did  you  look  at  the  false  report  statistics  that  came 
out  of  the  "Have  Cracks"?  And,  if  you  did,  can  you  make 
some  statement  about  what  the  probability  of  false 
reports  were  and  what  that  means? 

MR.  RUMMEL: 

On  false  reports*,  one  of  the  computer  dumps  that  we 
asked  for  and  just  got  this  morning  was:  What  is  the 
false  call  frequency  for  the  upper  10  percent  perform- 
ers versus  the  lower  10  percent  performers?  What  it 
looks  like  to  me  is  we  have  about  equivalent  situations 
in  this  case.  And  that  is  different  than  other  programs 
have  come  up  with.  I would  attribute  that  to  the 
greater  sampling  of  the  population.  But  we  looked  at  it 
for  just  one  test  specimen. 

MR.  CAUSTIN: 

Were  the  false  reports  screened  by  the  supervisor  like 
you  would  normally  do,  or  this  was  just  raw  data: 

MR.  RUMMEL: 

No.  In  addition  to  that  I think  that  in  treating  -- 
well,  the  first  of  all  this  dump  on  the  false  reports 
had  not  been  cleansed  in  terms  of  constructive  data.  So 
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there  may  be  some  error  there.  Secondly,  the  outliers 
have  not  been  cleansed  so  that  in  one  case  the  man 
found  more  false  call-outs  than  were  possible 
decisions. 

MR.  CAUSTIN: 

My  point,  though,  is  it  was  not  cleansed  like  you 
normally  do,  so  the  data  in  the  computer  can  mislead 
you.  And  as  a result,  I don't  agree  that  you  should 
penalize  for  false  reports.  You  are  going  to  get  that 
when  you  are  working  in  high  sensitivities.  You  have 
got  to  screen  them.  And  then  you  have  got  to  recognize 
the  data  . 

MR.  RUMMEL: 

Okay.  For  those  of  you  that  did  not  hear,  Ed  does  not 
agree  that  we  should  penalize  for  false  reports.  I, 
personally,  don't  either.  Part  of  the  rationale  for 
that  is  at  high  or  critical  performance  areas  where  we 
have  a lower  signal-to-noise , a false  call  incidence  is 
going  to  increase.  And  that  has  to  be  part  of  the  risk 
that  we  take  when  we  perform  that  inspection.  It  has  to 
be  part  of  the  experiment.. 

Joe . 

UNIDENTIFIED  SPEAKER: 

The  discussion  several  people  have  made  about  the 
Hawthorne  effect  or  whatever,  I have  always  been  of  the 
opinion  that  if  you  are  going  to  test  people  it  is  best 
to  let  them  know  because  if  you  don't  do  that  you  won't 
know  what  effect  it  has  on  the  final  results.  Maybe 
halfway  through  a test  you  will  get  people  who  will 
figure  it  out.  And  then  all  of  a sudden  your  test  has 
changed.  You  don't  know  when  it  changed  because  they 
may  figure  it  out  in  the  middle.  You  don't  know  if  some 
people  figure  it  out  and  some  don't.  All  of  a sudden 
you  might  have  two  distributions  right  in  the  middle. 

A second  comment  on  the  90/95,  I think  the  committee 
said  that  you  should  do  it  because  if  you  don't  do  it 
in  the  front  you  are  going  to  have  to  do  it  someplace 
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down  the  line.  That  does  not  to  me  address  the  question 
of  whether  95  is  realistic  at  any  place.  Whether  really 
we  should  just  cut  away  the  95  all  through  the  program. 
And  I think  the  answer  I heard  was  that  if  we  don't  put 
the  95  on  in  the  beginning,  we  are  going  to  have  to  put 
it  on  in  the  back  someplace.  I am  not  sure  that  the  95 
is  really  necessary. 

MR.  RUMMEL: 

That  incidentally  was  the  only  rationale  we  came  up 
with  for  keeping  it.  It  is  one  of  those  things  that 
everybody  feels  we  should,  but  is  a little  soft  in 
terms  of  rationale. 

UNIDENTIFIED  SPEAKER: 

With  respect  to  not  penalizing  for  false  calls  what  do 
you  do  when  the  probability  of  a false  call  is  higher 
than  the  probability  of  detection?  Is  that  a potential? 

MR.  BOISVERT: 

Very  much  so. 

MR.  RUMMEL: 

That  is  a potential.  That  says  the  signal  to  noise  is 
very  unfavorable.  And  you  are  expecting  because  you  are 
in  serious  trouble.  You  know  the  inspection  is  not 
going  to  be  very  reliable  but  you  are  willing  to  take 
the  results  of  what  you  get. 

UNIDENTIFIED  SPEAKER: 

Is  it  kind  of  like  the  crying  wolf? 

MR.  RUMMEL: 

It  can  be  crying  wolf.  But  when  you  get  into  that 
situation  you  have  to  go  into  it  with  your  eyes  open. 
And  you  are  usually  in  trouble  to  start  out  with.  The 
risk  has  been  made  and  you  will  take  it  consciously  or 
unconsciously. 
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UNIDENTIFIED  SPEAKER: 

I was  one  of  those  who  attended  one  of  your  sessions. 
What  I think  some  of  us  tried  to  get  across  was  not 
penalize  false  reports  period,  but  penalize  those  who 
have  repetitious  false  reports  just  because  of  their 
tendency  to  play  it  safe. 

MR.  RUMMEL: 

Okay.  The  gentleman  has  a statement  that  we  should 
penalize  false  reports  in  a long-term  management  sense 
based  on  the  inability  of  a guy  to  make  a decision  or 
his  decisiveness  as  opposed  to  the  capability  of  the 
test.  My  opinion  is  that  you  can't  do  that  because  you 
don't  have  the  right  measurement  feedback  in  your  own 
system . 

MR.  CAUSTIN: 

Well,  it  depends  on  what  your  outcome  was. 

MR.  RUMMEL: 

Right . 

UNIDENTIFIED  SPEAKER: 

Is  the  data  in  a format  that  will  allow  you  to  call  out 
if  you  went  to  a base,  for  example,  and  were  there  for 
a month  or  so  to  look  at  the  results  as  a function  of 
time  to  see  whether  it  peaks  early  and  decreases  or 
comes  out  as  a -- 

MR.  RUMMEL: 

The  question  is  will  the  data  base  allow  us  to  look  at 
performance  capability  as  a function  of  time  in  this 
case  calendar  date  to  try  to  put  to  bed  the  Hawthorn 
effect . 

Is  that  reasonably  the  question?  Okay. 


237 


J 


I 


MR.  BOISVERT: 

The  answer  is  yes. 

MR.  CAUSTIN: 

Apparently  you  had  a vote  in  this  No.  3 item,  could  you 
do  this  job  without  an  inspector  knowing  he  is  being 
tested.  Apparently  there  was  a vote.  I say  you  can  and 
it  has  been  demonstrates  that  you  can.  I was  just 
wondering  of  those  people  who  voted  how  many  have  tried 
it . *■ 


knowledge  of  test, 
group  probably  less 


MR.  RUMMEL: 

The  question  is  on  the  inspector's 
how  many  have  tried  it.  And  in  that 
than  10  percent  have  tried  it. 

UNIDENTIFIED  SPEAKER: 

There  was  an  industrial  test  -- 

MR.  RUMMEL: 

The  question  is:  We  had  a gentleman  in  session  two  of 
this  task  group  who  had  conducted  an  industrial  test. 
There  was  an  industrial  test  done  in  which  a part  was 
put  through  the  line  so  all  the  other  parts  looked 
alike. 

MR.  BOISVERT: 

We  still  need  the  question  repeated. 

UNIDENTIFIED  SPEAKER: 


two  we  have 
that  at  our 
to  prove  to 


It  was  a statement  that  in  the  session 
talked  about  this.  I had  made  the  comment 
factory  not  to  test  the  inspector  but  just 
the  engineers  a point  about  fracture  critical  parts, 
that  you  don't  play  games  with  inspectors,  that  you 
identify  the  parts  as  fracture  critical  so  the  people 

a hot  part  that  they  have  got 
To  prove  that  to  ourselves  we 
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series  fatigue  cracked  samples,  about  28  of  them 

that  had  a population  of  about  27  cracks  — not  in  all 
the  samples,  but  like  some  samples  had  three  cracks, 
some  had  none  — and  we  just  put  them  with  a fab  order, 
sent  them  down  to  the  shoes  for  inspection.  They  found 
none  of  the  cracks.  We  simply  cleaned  the  specimens, 
put  a critical  parts  tag  on  them,  sent  them  back.  And 
they  found  26  out  of  27. 

UNIDENTIFIED  SPEAKER: 

Is  that  a different  procedure? 

UNIDENTIFIED  SPEAKER: 

Same  procedure,  same  people,  same  process. 

MR.  CAUSTIN: 

In  our  shop  it  is  a different  proceduure  for  fracture 
critical  parts. 

UNIDENTIFIED  SPEAKER: 

Yes,  but  we  had  not  started  fracture  critical.  All  we 
did  was  stick  on  this  red  tag  that  said  these  are 
fracture  critical  parts  and  the  same  inspector,  same 
process,  everything  the  same.  The  only  thing  that  was 
different  was  the  tag. 

UNIDENTIFIED  SPEAKER: 

In  the  case  of  false  calls  on  Specimen  D which  later 
turns  out  through  destructive  testing  to  have  no 
cracks,  was  that  a matter  of  signal-to-noise  problem  or 
was  that  a matter  of  misinterpretation  of  strong 
signals  from  other  reflectors? 

MR.  RUMMEL: 

It  is  the  same.  That  is  the  same. 

UNIDENTIFIED  SPEAKER: 

I recognize  that  is  the  same.  But  in  a sense  misinter- 
pretation of  a signal  that  has  a stong  signal-to-noise 
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ratio  is  an  error  in  interpretation  and  more  of  a human 
factor  problem  than  it  is  of  an  equipment-type  related 
problem. 

MR.  RUMMEL: 

So  it  is  a pattern  recognition  problem. 

UNIDENTIFIED  SPEAKER: 

Were  false  calls  included  in  the  calculation  of  the 
curves  that  were  presented  Tuesday? 

MR.  RUMMEL: 

The  answer  to  that  is  no.  The  false  calls  are  reported 
in  tabular  form,  but  they  are  not  part  of  the  curve 
calculation.  No  one  knows  how  to  handle  it  yet  that  I 
am  aware  of . 

UNIDENTIFIED  SPEAKER: 

Then  you  are  really  loading  the  curve? 

MR.  RUMMEL: 

Yes  . 

UNIDENTIFIED  SPEAKER: 


It  seems  that  in  this  case  you  have  the  benefit  of 
destructive  testing  after  the  test.  Was  the  data 
analyzed  for  what  they  would  expect  to  have  found 
without  destructive  testing  versus  what  they  would  have 
found,  what  they  did  find  when  they  did  destructive 
test  the  parts?  Well,  in  other  words,  you  are  doing 
nondestructive  testing.  Your  nondestructive  testing 
criteria  are  established  in  some  way,  usually  on  the 
basis  of  some  probability  from  some  destructive 
testing . 
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certain  location  where  you  wouldn't  always  have  that 
had  you  not  destructively  tested  the  parts  after  the 
test  series. 

UNDENTIFI ED  SPEAKER: 

Well,  no  secondary  backup  procedure  was  used  whereas  if 
they  were  arranged  in  normal  Air  Force  procedure  — 

UNIDENTIFIED  SPEAKER: 

In  other  words,  most  of  the  time  in  your  testing  you 
may  reject  a part  or  accept  a part  from  an  indication 
that  you  assume  is  either  false  or  inaccurate  and  it 
may  or  may  not  be. 

MR.  RUMMEL: 

I think  I understand  the  question. 

We  sometimes  will  reject  since  our  ability  to  specify 
what  we  are  going  to  see  and  what  is  acceptable  in  a 
penetrant  area  may  not  be  precise.  And  I think  that  is 
different  from  the  programs  that  we  are  trying  to 
measure.  That  almost  gets  into  the  lawyer  area. 

Other  comments,  Bernie? 

MR.  BOISVERT: 

Yes.  Hey,  we  can  continue  this  all  day.  I would  like 
to.  We  are  running  way  behind  time. 

MR.  CAUSTIN: 

I am  getting  the  impression  that  people  feel  false 
calls  are  bad.  But  take  another  look  at  it.  That  is  a 
measure  of  conservatism  of  the  process.  And  it  is  more 
deleterious  to  the  object  to  miss  something  that  you 
should  find.  It  is  a measure  of  conservatism.  Don't 
look  at  it  as  a negative. 

MR.  RUMMEL: 

One  additional  thing  I thought  was  interesting  in  that 
respect  was  one  fellow  said  that  false  calls  were  good 
because  they  keep  the  operator  alert. 
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MR.  BOISVERT: 


We  are  running  way  behind  time.  How  about  if  we  do  this 
on  reliability  measurement  and  modeling,  we  will  delay 
additional  comments  until  the  very  end  of  the  meeting. 
Those  of  us  that  want  to  continue  this  discussion  — 
and  I am  one  --  we  will  remain  here  after  we  officially 
adjourn.  Right  now  what  we  would  like  to  do  is  take 
about  a five-minute  relief  break,  okay. 


242 


VII 


FRACTURE  MECHANICS/NDI  INTERRELATIONSHIPS 


TASK  GROUP  LEADERS: 

W.  L.  Castner 
E.  L.  Caustin 
M.  A.  Owen 

MR.  BOISVERT: 

The  next  group  concerns  the  interface  or  the  inter- 
relationship between  the  structural  engineer,  the  guy 
doing  the  data  damage  tolerance  analysis  and  the  NDI . 
To  help  us  in  this  area  I think  we  have  got  perhaps  the 
outstanding  group,  Bud  Castner  from  NASA,  Johnson  Space 
Center,  Ed  Caustin,  and  one  of  the  guys  that  really  is 
honchoing  the  whole  thing,  Mark  Owen  out  of  our  struc- 
tural engineering  group  at  Wr ight-Patterson . 

Mark. 

MR.  M.  A.  OWEN,  Summary  on  Fracture  Meehan i cs/NDI 
Interrelationship:  As  Bernie  mentioned  we  were  looking 
at  some  of  the  interrelationships  between  this  area  of 
fracture  mechanics  and  NDI.  I am  happy  to  report  that 
we  did  find  that  there  are  some  interrelationships.  On 
behalf  of  Bud  and  Ed,  I would  like  to  thank  those  that 
participated  in  our  task  group.  We  had  some  good 
participation,  good  discussions.  But  I feel  a little 
bit  like  a saying  that  a friend  of  mine  has:  We  came, 
we  saw,  and  two  out  of  three  is  not  bad. 

We  did,  however,  reach  some  concensus.  I think  in  order 
to  present  the  things  that  we  were  able  to  conclude  we 
will  just  proceed  into  the  items  that  the  three  of  us, 
the  three  task  leaders,  decided  that  were  consensus  of 
all  the  people  that  met  with  our  specific  group. 

One  that  came  through  very  clearly  is  in  the  area  of 
communication.  And  it  was  broken  down  into  three 
different  aspects  of  communication.  That  communication 
I am  talking  about  is  communicating  problems  and 
lessons  learned.  One  is  that  the  Air  Force  has  a 
considerable  amount  of  information  in  the  fracture 
mechanics  area  specifically  that  is  not  being  done  in  a 
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damage  tolerance  assessment  of  old  airplanes  that  is 
not  really  being  communicated,  not  a conscientious 
effort  being  made  to  communicate  to  all  other  services, 
to  industry.  So,  we  need  to  make  an  effort  on  the  Air 
Force  side  to  improve  that  kind  of  communication.  The 
NDI  people  are  probably  doing  a little  better  job  on 
the  Air  Force  side  by  having  meetings  like  this.  The 
communication  between  fracture  mechanics  people  and  NDI 
people  needs  to  be  improved.  Each  has  their  own  unique 
set  of  problems.  And  there  is  a lot  of  misunderstanding 
between  those  two  sets  of  groups  about  what  those 
unique  problems  are. 


I think  those  of  us  involved  in  fracture  mechanics 
often  ask  the  impossible  of  the  NDI  people.  I think 
sometimes  the  NDI  people  ask  the  impossible  of  the 
fracture  mechanics  people.  They  want  to  know  exactly 
what  they  are  to  look  for,  exactly  where  it  is  going  to 
be,  exactly  what  it  is  going  to  look  like  when  they  see 
it.  We  need  to  improve  the  overall  communication  and 
process  there. 

One  thing  that  came  out  during  the  course  of  our 
session  is  that  there  is  a lot  of  industry  experience 
in  NDI,  programs  such  as  the  B-l , lessons  that  have 
been  learned  there  on  correct  procedures  to  use  in 
doing  NDI  that  are  really  not  being  transitioned  into 
tne  Air  Force  services. 

So,  that  communication  needs  to  be  improved. 

The  next  concensus  that  we  were  able  to  reach  is  that 
we  have  seen  a lot  of  statistics  that  show  that  there 
are  uncertainties  in  NDI.  There  are  also  some  uncer- 
tainties in  fracture  mechanics  as  well.  One  large  group 
of  uncertainties  is  in  the  entire  materials  data  base 
for  fracture  mechanics,  especially  in  some  of  the  older 
— as  we  began  to  analyze  some  of  the  older  airplanes 
with  not  so  recent  materials  where  we  didn't  look  at 
damage  tolerance  type  properties.  And  consequently  no 
data  were  generated  on  those  materials. 

A big  area  that  was  mentioned  is  as  the  Air  Force  moves 
in  to  look  at  flight  damage  tolerance  criteria  to  the 


245 


. .. 


engines,  is  that  whole  aspect  of  the  engine  environment 
and  the  impact  that  that  is  going  to  have  on  predicted 
lives  and  how  do  we  really  design  with  all  those  un- 
certainties that  have  to  do  with  temperature,  time, 
creep,  all  the  other  things  that  go  into  the  engine  and 
its  structure. 

We  also  reached  the  conclusion  after  some  amount  of 
debate,  I guess,  that  these  statistics  that  we  see  here 
and  that  came  from  the  "Have  Cracks-Will  Travel"  we  can 
do  better  than  that  in  specific  hot  spot  inspections. 
We  think  when  procedures  are  specifically  set  to 
analyze  or  look  at  one  specific  area  of  structure  that 
is  decreed  to  be  a hot  spot  and  especially  critical,  if 
it  is  not  a routine  type  of  inspection  that  people  go 
in  there  knowing  the  criticality  of  that  specific  area 
and  that  if  we  somehow  decide  who  the  best  inspectors 
are  and  then  pick  those  best  inspectors  to  do  that 
specific  inspection,  then  the  reliability  numbers  that 
you  would  see  for  those  specific  hot  spots  would  be 
better  than  what  we  see  in  the  "Have  Cracks"  effort. 

Some  more  concensus  items.  With  respect  to  NDI  relia- 
bility demonstration  programs  we  about  concluded  that 
those  type  of  programs  would  not  be  very  practical  or 
indeed  very  helpful  for  routine  inspection  on  aircraft. 
There  are  two  reasons  I guess  that  we  see  that.  This  is 
specifically  with  respect  to  implementing  those  ten  or 
fifteen,  the  latest  Air  Force  criteria.  Number  one,  in 
design  of  new  aircraft  we  are  not  depending  on  in 
general  the  fleet  inspections,  the  in-service  inspec- 
tions to  protect  the  airplanes.  We  are  treating  a lot 
of  parts  as  non- inspectable  and  trying  to  design  in 
enough  damage  tolerance  so  that  we  don't  have  to 
perform  a lot  of  inspections. 

Another  aspect  to  that,  not  practical  for  routine 
inspections,  and  I forgot  what  it  is  right  now.  I will 
come  back  to  it. 

However,  as  opposed  to  routine  inspections,  there  are  a 
lot  of  hot  spot  inspections  that  we  are  in  the  process 
of  having  to  do  on  some  of  our  older  airplanes,  notably 
the  T-38  as  was  cited  yesterday.  And  I guess  we  all 
agreed  that  in  addition  to  having  the  best  inspectors 
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and  having  it  not  routine  and  spelling  out  special 
procedures  that  it  also  would  be  desirable  to  have  a 
demo  program  for  those  hot  spot  inspections  to  improve 
the  overall  confidence  that  we  have  a higher  relia- 
bility of  finding  the  things  that  we  need  to  find.  It 
was  pointed  out,  however,  that  it  may  not  be  --  hot 
spots  come  up  all  of  a sudden  sometimes.  And  you  don't 
have  a lot  of  time  to  put  together  a demo  program 
before  you  go  to  the  inspections.  So  that  is  kind  of 
the  other  side  of  the  coin. 

We  also  concluded  that  looking  at  these  reliability 
statistics  that  came  out  of  "Have  Craks,"  we  concluded 
apparently  what  the  reliability  group  concluded  that 
there  needs  to  be  some  trade-off  studies  run  with  those 
reliability  results.  This  specifically  came  up  in  the 
area  when  we  discussed  the  90-percent  probability  with 
95-percent  confidence  level  criteria. 

There  are  a lot  of  other  ways  to  achieve  that  same 
level  of  reliability.  A couple  of  those  other  ways  are 
with  more  frequent  inspections  or  with  multiple  inspec- 
tion techniques.  And  it  would  be  an  interesting  course 
of  study  to  use  these  statistics  that  came  out  of  "Have 
Cracks"  and  put  all  those  numbers  together.  That  is  a 
requi r ement . 

The  other  one  that  was  not  too  practical  for  routine 
inspections,  NDI  demonstration  is  that  personnel  keep 
getting  moved  around  quite  a bit.  In  the  Air  Force  we 
don't  keep  the  same  inspector  all  the  time  and  continue 
the  update  process.  Sometimes  the  inspections  are  kind 
of  done  all  over  the  territory.  Now,  when  we  say  NDI 
demo,  that  is  not  the  same  thing  as  certification.  I 
hope  everyone  understands  that. 

The  other  area  that  we  concensus  was  in  the  area  of  the 
presentation  that  was  made  with  respect  to  the  screen- 
ing samples.  That  whole  area  of  screening  samples,  the 
reliability  that  came  out  of  screening  samples  versus 
the  reliability  that  comes  out  of  actual  structure, 
that  whole  area  has  got  to  be  resolved.  The  ultrasonic 
inspection  showed  the  same  trend,  but  there  was  no 
correlation  at  all  with  the  eddy  current  inspections. 
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So  that  has  to  be  resolved 
dramatic  implications  even  to 


because  that  has  pretty 
an  NDI  demo  program. 


Then,  we  finally  came  to  the  conclusion  that  on  the 
basis  of  these  "Have  Cracks"  numbers  that  routine  NDI 
cannot  be  relied  on  to  protect  flight  safety  for 
generalized  aircraft  cracking.  And  the  example  that  we 
cited  there  was  the  KC-135  where  cracks  could  be  any 
place  in  the  wing  or  cover,  these  statistics  tell  us 
that  we  could  not  protect  the  flight  safety  of  that 
airplane  by  normal  routine  inspections.  We  have  to  come 
up  with  the  other  means  to  protect  flight  safety  at 
this  point  in  time. 


Now,  of  course,  there  were  some  other  questions  that  we 
didn't  really  reach  a concensus  on.  One  that  several 
people  discussed  was:  is  90/95  realistic?  I guess  on 
the  basis  of  these  statistics  we  concluded  that  it  is 
not  realistic  for  routine  inspections  if  that  is  what 
these  statistics  are  representative  of.  Maybe  it  is  not 
realistic  or  even  necessary  is  the  other  thing  we  con- 
cluded. However,  it  is  a reasonable  goal  for  fracture 
critical  parts.  For  parts  that  the  flight  safety  of  the 
airplane  depends  on  not  having  a crack  in  90/95  is  a 
reasonable  goal.  We  used  the  term  "reasonable  goal," 
which  doesn't  say  that  if  you  come  up  with  85  percent 
probability  and  95  percent  confidence  level  that  that 
is  bad.  90/95  is  a reasonable  goal. 


And  another  question  that  there  was  considerable 
interest  in  is:  Are  the  production  NDI  demonstration 
programs  that  have  been  run,  like  on  the  B-l  and  A-10 
and  others,  are  they  really  applicable  to  assembled 
aircraft?  The  only  thing  we  could  conclude  about  that 
is  that  we  don't  even  know  of  a study  that  answers  that 
question.  It  is  an  interesting  question,  however.  So, 
of  all  that  concensus  and  the  unanswered 
came  up  with  this  list  of  recommended 
is  in  the  area  of  reliability  trade-offs, 
a study  with  respect  to  production  NDI 
applicability.  We  asked  ourselves  the 
would  Air  Force-type  people,  the  same 
field  level  type  people  have  done  with  a production  NDI 
demo-type  program,  what  reliability  would  have  come  out 
of  that.  And  on  the  other  hand,  how 


on  the  basis 
questions  we 
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The  other  is 
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would  production  people  have  done  with  the  "Have 
Cracks"  type  of  study? 

We  did  not  know  the  answer  to  that.  So  that  would  be  an 
interesting  course  of  study. 

Another  concern  that  came  up  was:  Have  the  inspection 
procedures  been  detailed  and  optimized  enough  for 
fracture  critical  parts? 

In  other  words,  the  procedures  that  were  identified  in 
the  "Have  Cracks"  Program,  were  they  really  optimized 
if  you  were  inspecting  fracture  critical  parts?  There 
might  be  some  way  to  further  optimize  those  procedures. 
For  instance,  one  point  that  came  up  was  if  you  cleaned 
bolt  holes  prior  to  eddy  current  inspection. 


Finally  we  concluded  that  with  respect  to  one  of  the 
last  questions  we  were  asked  to  look  into;  that  is, 
methodologies  for  cranking  all  these  results  together, 
inspection  of  reliability  models,  flaw  population 
models.  There  are  some  techniques  to  do  that,  and  those 
are  in  the  form  of  risk  analysis.  There  have  been  such 
studies  run  on  the  B-52,  the  KC-135,  and  others.  What 
we  are  suggesting  is  it  would  be  an  interesting  course 
of  study  to  take  the  "Have  Cracks"  numbers  and  crank 
that  into  a risk  analysis  to  see  what  type  of  sensi- 
tivities and  economic  answers  one  might  come  up  with. 


Conclusions:  We  think  new  damage  tolerance  designs  are 
going  to  alleviate  somewhat  this  NDI  Problem.  We  are 
going  to  be  shooting  for  less  field  inspections  to 
protect  safety  of  flight.  We  are  going  to  design  in 
safety  of  flight  from  the  beginning  throughout  the  life 
of  the  airplane. 


However,  we  don't  expect 
go  away.  We  are  still 
design  concepts  that  we 
people  are  going  to  have 
soon  as  we  get  a new  flee 
aircraft  in  the  field. 


all  the  hot  spots  are  going  to 
stuck  with  some  of  the  same 
used  to  use.  So  all  the  NDI 
to  pack  up  their  equipment  as 
t of  damage  tolerance  designed 
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er , I don' t know  how  anybody  could  look  at  the 
Cracks"  data  and  not  conclude  that  the  in-service 


NDI  needs  to  be  improved.  On  the  other  hand  we  asked 
ourselves  one  last  question  that  we  would  like  to  have 
you  ask  yourself.  And  that  is:  How  would  the  other 
aircraft  disciplines  besides  NDI  fare  under  the  same 
kind  of  scrutiny  that  has  been  given  the  NDI  program  at 
this  point  in  time.  I don't  quite  know  the  answer  to 
that  question,  but  it  is  worth  thinking  about. 

That  is  all  I have.  I would  be  happy  to  entertain  any 
questions  you  may  have  or  any  comments,  additions,  or 
corrections  that  anybody  who  particpated  in  our  group 
would  like  to  bring  up  at  this  time. 
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VIII 


DATA  ANALYSIS 


TASK  GROUP  LEADERS: 

D.  E.  Pettit 
J.  A.  Moyzis 
R.  R.  Wagner 

Mr.  J.  A.  Moyzis,  Summary  on  Data  Analysis:  Okay.  We 
are  starting  off  with  something  we  have  heard  about 
already.  Maybe  we  have  had  more  discussion  on  this  than 
the  whole  subject  merits  in  this  conference.  But  let  me 
just  go  over  these  points  that  we  have  come  to,  the 
conclusions  we  have  come  to.  The  first  one  that  we 
looked  at  was  false  calls  and  analytic  treatment  in  the 
reliability  data.  Really  what  I would  like  to  say  that 
is  not  very  important  but  there  are  so  many  issues  that 
are  really  important  in  the  data  that  we  have  collected 
so  far  that  unless  costs  are  a prime  consideration  -- 
and  we  have  heard  some  of  that  in  the  production 
environment  --  but  unless  costs  become  an  overriding 
consideration,  maybe  this  topic  is  not  as  important  as 
a number  of  other  topics. 

Now,  as  far  as  how  you  analytically  treat  --  cne 
suggestion  arose  during  the  discussion  that  one  way  of 
handling  the  data  is  to  try  to  correlate  the  percentage 
of  false  calls  that  an  individual  generated  to  his 
performance  overall,  whether  he  was  a good  inspector 
overall  or  a bad  inspector  overall  or  whether  he  was  a 
high  false  call  rate  meant  or  correlated  or  was  related 
to  the  fact  that  he  had  a good  performance  on  small 
flaw  sizes,  which  might  indicate  that  he  is  looking  for 
everything  and  he  is  calling  the  small  flaws  but  he  is 
also  calling  things  that  aren't  there. 

But  as  far  as  analytically  folding  it  into  the  curve, 
there  was  not  any  concensus  at  all  to  do  that. 

The  third  bullet  there  is  that  the  feeling  was  that  if 
somebody  wants  to  do  an  analytic  treatment  --  and  as  I 
guess  I have  already  said,  nobody  that  we  heard  in  our 
discussions  in  all  three  discussions  was  in  favor  of 
folding  this  data  in  analytically.  And  I seem  to  hear 
from  another  group  that  there  was  some  rumbling  about 
that.  But  we  did  not  hear  it  in  our  group. 
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If  there  is  such  a desire  to  do  that,  there  better  be 
some  mathematically  rigorous  analytic  treatment  that 
everbody  will  buy  rather  than  the  kind  of  thing  I think 
college  professors  do  on  multiple  choice  questions: 
Well,  I will  arbitrarily  take  two  points  off  for  a 
wrong  answer  and  get  some  sort  of  a score.  So,  rather 
than  any  kind  of  a hokey  analytic  scheme,  if  you  are 
going  to  do  it,  it  better  be  something  that  has  a 
mathematical  rigor. 

The  concensus  overall  on  this  false  call  was  that  don't 
analytically  fold  it  into  the  data.  And  that  was 
uniform.  I was  surprised.  I really  expected  and  I have 
heard  from  other  people  in  the  past  personally  that  you 
should.  And  in  this  group  it  was  uniform,  don't.  But 
there  was  a very  definite  feeling  like,  don't  ignore 
the  data.  You  want  to  have  this  kind  of  data  available 
so  you  know  who  the  people  are  who  are  giving  you  the 
false  calls.  Maybe  that  is  a cry  of  wolf.  But  if  you 
find  a guy  who  cries  wolf  every  once  in  a while,  he  is 
also  finding  small  defects,  too.  Now,  that  is  really 
all  we  came  up  with  on  false  calls. 

Now,  an  important  thing  that  came  up  in  our  discussions 
was  this  idea  of  our  95-percent  confidence  levels.  The 
point  was  raised,  and  I think  is  a very  good  one,  there 
is  a lot  of  scatter  in  the  data,  and  when  you  look  at 
the  scatter  in  the  data  one  thing  right  away,  you  are 
not  going  to  get  a good  90/95  percent  confidence  level 
just  by  collecting  more  data.  So,  you  somehow  say, 
"Well,  the  95-percent  confidence  level  is  low  because 
you  don't  have  a lot  of  sample  points."  And  I have  said 
that  in  the  past  myself.  But  when  you  look  at  the  data 
and  it  has  that  much  scatter,  if  you  collected  10  times 
more  data  that  has  that  amount  of  scatter,  maybe  your 
95  will  shift  a little  but  it  is  not  really  going  to 
get  up  into  the  90  percent  level  and  answer  the 
question  of  what  crack  level  can  we  find  things  to 
90/95.  So,  that  comes  out  that  just  collecting  more 
data  of  this  kind  is  not  going  to  help. 

Hie  second  thing  is:  What  do  you  do  about  the  scatter? 
Why  is  there  scatter?  One  thing  is  that  it  may  just  be 
inherent  in  the  data.  You  may  just  have  that  kind  of 
thing  inherently.  But  a couple  of  suggestions  of  how  we 
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might  investigate  whether  that  scatter  can  be  reduced 
is  to  look  — we  do  plot  versus  crack  length.  Is  the 
crack  length  the  meaningful  parameter?  Is  there  another 
parameter  that  we  could  plot  the  reliability  against 
that  might  reduce  the  scatter?  That  could  be  one  way. 

Another  one  is  the  question  of  any  time  you  do  a least 
squares  fit  or  any  kind  of  a fit,  if  you  look  in  the 
statistics  book  there  are  all  the  conditions  that  the 
data  is  supposed  to  satisfy  so  that  the  least-square 
fit  would  really  mean  something.  In  a linear  regression 
you  get  the  thing  that  this  formula  is  good  for  if  you 
have  a distribution,  a normal  distribution  around  a 
mean  at  every  "X”  value,  and  that  normal  distribution 
is  the  same  for  every  "X"  value,  that  the  deviation 
from  the  mean  is  the  same  all  the  way  through.  Is  that 
true  for  this  data?  I think  we  have  got  enough  data 
where  we  can  start  looking  at  some  of  this.  We  can  pick 
a crack  length  and  take  a look  at  how  the  data  is 
scattered  and  take  a look  at  whether  it  is  uniform 
along  the  way.  And  also  the  question:  Is  all  the  data 
that  we  are  using  from  the  same  distribution. 

Are  there  a uniform  set  of  inspectors?  Are  they  poor 
inspectors?  Are  they  good  inspectors?  So  we  end  up  with 
at  any  crack  size,  not  a normal  distribution  but  maybe 
a bi-normal  or  tri-normal  or  multi-normal.  Maybe  we 
could  reduce  the  scatter  by  breaking  it  up  a little. 
The  point  is,  I think,  overall,  that  unless  you  can 
reduce  the  scatter,  the  question  we  would  like  to  ask 
and  we  historically  started  out  to  ask  about  90/95  was: 
Where  is  that  magic  number  that  I can  guarantee  I can 
find  all  cracks  above.  We  can't  find  that  collective 
data  this  way.  And  that  was  something  that  came  up. 

The  next  story  that  came  up  --  and  it  is  a one  liner  -- 
is  that  there  has  to  be  a great  deal  of  recognition  in 
the  fact  that  statistics  does  not  give  you  cause  and 
effect,  it  gives  you  relationships.  I wish  I would  have 
said  that.  But  somebody  did.  When  you  see  a relation- 
ship you  have  got  to  be  very  careful  about  what  that 
means.  That  does  not  mean  that  because  the  eyeglasses 
and  reliabilty  somehow  show  up  with  some  sort  of  a 
relationship,  that  doesn't  mean  that  because  you  take  a 
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normal  operator  and  poke  him  in  the  eye  so  he  has  to 
wear  glasses,  he  will  get  better.  That  kind  of  a thing 
has  to  be  said.  It  is  just  a one  liner,  and  it  has  to 
go  along  with  everything  else  we  see  here. 

Another  point  that  was  brought  up  was  with  the  amount 
of  data  we  have  could  possible  start  to  do  some 
multi-variant  analyses  with  the  data.  Now,  I was  given 
some  key  phrases:  Aid  analysis,  which  I know  something 
about:  progrit,  which  I know  nothing  about.  There  was  a 
number  of  them  that  were  suggested.  But  here  is  an 
opportunity  with  the  number  of  parameters  we  have, 
instead  of  plotting  and  anlyzing  things  as  a function 
of  one  parameter  maybe  we  can  go  into  a multi-parameter 
analysis.  One  is  to  test  the  interaction  of  the 
variables,  we  can  maybe  see  whether  some  of  these 
relationships  are  really  meaningful.  Maybe  it  is 
correlated  or  related  to  some  third  thing  that  we 
should  bring  in.  And  also  maybe  a multivariant  analysis 
would  bring  out  the  fact  of  whether  you  have  all  the 
important  parameters,  you  have  collected  all  the 
important  parameters.  There  was  a definite  feeling  by  a 
number  of  people  --  you  can  collect  parameters  forever. 

But  there  are  possibilities  that  the  psycholog ical 
makeup,  that  nebulous  thing  that  a guy  carries  on 
inside  himself  may  be  the  most  important  thing.  It  has 
nothing  to  do  with  his  age  or  glasses  or  hair  color  or 
whatever.  Maybe  a multi-variant  analysis  cannot  answer 
that.  What  important  parameters  do  you  leave  out?  Maybe 
it  will  point  to  the  fact  that  there  are  parameters  in 
there  that  you  have  not  paid  attention  to.  We  can't 
tell  you  what  they  are,  but  maybe  you  should  go  looking 
for  them. 

Now,  another  point  that  was  brought  out  in  the  data 
analysis  is  that  the  type  of  data  analysis  you  do  is 
very  dependent  on  what  you  are  going  to  use  the 
reliability  results  for.  Now,  historically  I think  that 
the  reliability  data  that  has  been  collected  --  it 
started  out  sort  of  an  NDI  fracture  mechanics 
relationship.  But  since  I have  been  in  on  the  thing 
anyway  it  said  90/95  question  where  is  the  largest 
crack  you  can  find. 


When  you  ask  that  question,  you  collect  the  data  in  the 
way  that  we  have  been  collecting  it.  You  might  try  to 
fit  with  a curve  where  the  95-percent  confidence  level 
crosses  the  90  level.  Now  you  can  say  I know  where  it 
is  and  try  to  draw  a smooth  curve.  Maybe  we  are  wrong 
in  doing  that.  There  are  a lot  of  different  functions 
that  your  reliability  data  might  want  to  assess;  90/95, 
cost/risk  analysis,  NDI/fracture  mechanics,  facility 
comparison.  Somebody  brought  up  the  point  that  maybe 
you  should  be  giving  a standard  statistical  question  to 
the  data.  You  can  make  the  hypothesis  that  the  proba- 
bility of  detection  by  the  depots  is  equal  to  the 
probability  of  detection  in  the  field.  You  can  test 
that  hypothesis.  That  is  a different  kind  of  treatment 
of  the  data.  You  can  do  it  at  any  single  crack  length. 

So,  if  you  are  going  to  analyze  the  data,  if  you  are 
going  to  use  a certain  model,  you  have  to  keep  in  mind 
what  you  want  to  do  with  it.  That  is  where  you  bring 
the  statistician  in.  You  say:  This  is  the  kind  of 
question  I want  to  answer,  how  should  I collect  data, 
what  should  I do  so  that  I have  a possibility  of 
answering  that  question. 

The  recommendations  that  we  were  able  to  glean  out  of 
this  was  that  there  is  not  a lot  of  reason  to  go  on  and 
start  collecting  more  data  until  we  fully  analyze  the 
present  data  that  is  available.  That  takes  the  Air 
Force  off  the  hook  for  more  money.  If  you  analyze  the 
data  that  is  available  that  will  allow  you  perhaps  to 
define  thee  questions  that  you  want  the  further  data  to 
answer.  You  have  got  a certain  amount  of  data  now,  and 
you  can  milk  it  for  all  it  is  worth  and  try  to  come  up 
with  those  questions  you  would  like  to  have  answered 
and  see  which  ones  are.  If  you  find  some  that  are 
important  that  are  not  answered,  it  gives  you  a motiva- 
tion to  go  out  and  fill  a gap  of  that  part  of  the  data. 
You  can  direct  the  kind  of  data  that  you  are  going  to 
get . 

If  you  have  a very  specific  question  that  you  would 
like  to  answer  that  not  only  would  allow  you  to  see  the 
kind  of  data  you  want  to  collect,  but  beforehand  you 
could  lay  out  the  models.  This  is  how  I am  going  to 
analyze  the  data  when  I finally  get  it.  I am  going  to 
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do  it  this  way  because  this  is  the  kind  of  question  I 
want  to  answer.  It  will  also  allow  you  to  design  the 
experiment  to  elucidate  cause  and  effect,  in  other 
words,  to  try  to  get  closer  to  that  cause  and  effect 
rather  than  coming  up  with  some  relationship. 

How  are  you  to  design  the  experiment  so  that  you  can 
have  a stronger  feel  that  there  really  was  a link 
between  Parameter  A and  Parameter  B?  And  something  I 
think  that  maybe  as  we  were  putting  it  together  we 
looked  at  this  is  when  you  do  this  you  would  like  to 
have  20-20  hindsight  at  the  beginning.  It  would  be  nice 
when  you  do  this  maybe  to  have  some  idea  of  if  I just 
asked  a question,  I lay  a model,  it  may  turn  out  when  I 
am  done,  two  or  three  or  four  years  down  the  line,  that 
the  world  is  changing.  Can  I try  to  guess  what  the 
world  is  changing.  Can  I try  to  guess  what  that  change 
may  be  so  that  when  I do  collect  the  data  I am  not  hung 
with  the  fact  that  I have  data  that  can  only  be 
analyzed  one  way  and  will  only  answer  one  kind  of 
question  and  it  turns  out  that  this  is  not  the  question 
we  are  interested  in.  That  is  an  impossible  thing  to  do 
in  any  kind  of  rigor.  But  maybe  when  you're  trying  to 
do  that  kind  of  a look  ahead,  you  should  at  least  try. 

A separate  recommendation  was  that  there  should  be  some 
concensus  on  the  handling  of  data  in  demonstration 
programs.  I think  the  demonstration  programs  up  to  now 
have  been  a range  type  of  analysis.  And  partly  I think 
that  is  done  because  of  cost.  The  90/95  costs  a lot  of 
money . 

So,  you  isolate  four  range  sizes  or  so.  Is  there  some 
way  that  m~ybe  demo  data  could  be  handled  where  we 
could  get  away  from  that  kind  of  expense? 

Now,  there  is  an  ASNT  document  that  has  been  floating 
around  for  a long  time  and  may  eventually  be  published. 
I would  undoubtedly  think  that  when  it  finally  gets  out 
the  people  will  say  what  the  heck  is  this  all  about.  I 
can't  buy  that  at  all. 

And  finally,  a suggestion  was  made  that  a glossary  of 
definitions  would  be  nice  so  we  all  know  what  we  are 
talking  about.  What  is  a confidence  level?  I think  if 


we  get  about  five  or 
have  between  two  and 
big  argument.  I think 
of  that.  I don't  know 
of  the  discussions  we 
disagree  with  others 
about  different  things 


ten  people  in  a room,  we  should 
five  different  definitions  and  a 
that  the  ASNT  document  tries  some 
how  extensive  that  would  be.  Some 
do  have  when  we  rant  and  rave  and 
is  because  we  are  really  talking 


That  is  all  I have  to  say. 
UNIDENTIFIED  SPEAKER: 


This  ties  into  your  last  thing  about 
think  one  thing  that  everybody  recogni 
your  noise  goes  up,  false  calls  go  up. 
talking  about  a false  call,  well,  is  ult 
a false  call  or  is  it  noise?  I think  we 
tion  of  what  noise  is  and  how  that  t 
calls. 


definitions.  I 
zes  is  that  as 
When  you  start 
rasonic  scatter 
need  a defini- 
ies  into  false 


MR.  MOYZIS: 

Okay.  A false  call  to  me  is  anything  off  of  nothing  or 
off  of  something  that  is  not  a crack. 

MR.  CAUSTIN: 


Some  of  the  data  that  I looked  at  on  this  test  that  was 
run  from  your  "Have  Cracks-Will  Travel  Program"  indi- 
cated that  it  was  not  run  with  fracture  mechanics  in 
mind  as  far  as  the  -36  manual  is  intended.  So,  your 
interface  with  fracture  mechanics,  you  are  really 
stretching  it  when  you  are  talking  about  90/95  because 
you  have  already  put  the  inspector  in  a bad  position. 
You  did  not  give  him  instructions  in  the  calibration. 
In  other  words,  those  disciplines  that  you  use  for  a 
fracture  critical.  And  also  the  high  number  of  false 
calls  --  and  I guess  you  must  have  gotten  an  enormous 
number  of  them  because  everyone  seems  so  concerned 
about  it  --  tells  me  that  there  also  must  have  been 
something  wrong  with  the  techniques  used.  False  calls 
are  another  look  at  the  techniques. 


For  example,  did  you 
looked  at  them  with 


clean  up  those  holes  before  you 
eddy  current.  Normally,  in  the 
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teardown  in  our  world  we  do  that  to  reduce  the  number 
of  false  calls.  So,  those  kind  of  things  tell  me  that, 
hey,  there  is  still  question  about  how  it  was  run 
before  you  take  a look  at  that  data  across  the  board  to 
make  a lot  of  answers. 

MR.  MOYZIS: 


Okay.  I don't  know  the  number  of  false  calls  that  were 
made.  I don't  think  this  data  was  collected  for 
fracture  mechanics  particularly.  It  was  collected  to 
see  how  well  are  we  doing  in  the  field.  We  have  gotten 
into  a lot  of  defining  things  on  how  we'll  do  on  90/95. 

UNIDENTIFIED  SPEAKER: 


There  were  a lot  of  false  calls.  There  were  a few 
people,  Ed,  that  did  have  a pr  eponderance  of  false 
calls.  But  they  were  the  exception  rather  than  the 
rule.  I would  say  generally  it  was  fairly  low. 

MR.  BOISVERT: 

I tried  to  point  that  we  put  this  program  just  like 

we  were  doing  four  years  ago.  We  have  improved  somewhat 
in  the  program.  But  this  is  what  we  are  doing  in  the 
field.  Now  you  make  the  point:  Do  we  clean  up.  No,  no. 
We  don't  have  time  to  clean  up  in  a maintenance 
environment  unless  it  is  a real  serious  condition. 
Normally,  we  go  out  and  inspect.  If  it  is  very 
critical,  then  we  do  go  in  and  inspect. 

MR.  CAUSTIN: 

If  you  shoot  for  90/95  that  is  critical. 

J MR.  BOISVERT: 

The  original  intent  was  to  find  out  where  are  we  today. 
That  was  four  years  ago.  Now  we  are  coming  up  with  this 
90/95.  It  is  a good  question,  but  the  original  intent 
was  to  find  out  where  we  are  today. 
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have  done  an  outstanding  job.  They  have  given  me  enough 
work  to  do.  I would  like  a big  hand  for  these  task 
group  leaders,  please. 

This  concludes  the  meeting. 


Section  VI 

LUNCHEON  ADDRESS  SUMMARY 


A summary  of  the  luncheon  address,  "The  Role  of 
Inspections  in  the  Maintenance  of  Aircraft  Safety"  by 
Mr.  C.  F.  Tiffany,  USAF  Air  Systems  Division,  Wright- 
Patterson  Air  Force  Base,  Ohio,  is  provided  to  exhibit 
the  technical  challenges  for  NDI . No  prepared  text  was 
available  for  this  summary  but  the  viewgraph  copies 
were.  The  commentary  is  therefore  derived  from  recol- 
lections of  the  talk  combined  with  information  from  the 
visual  aids.  Al 1 
d iag rams . Co p i e s o 1 
end  of  this  Section. 

Topic  Outline 


percentage  values  are  estimated  from 
> f the  v i e wg r a ph s are  included  at  the 


o Some  Aircraft  Accident  Statistics 
o Primary  Causes 

o Some  Basic  Aspects  of  Structural  Failures 
o Past  USAF  Approach  to  Aircraft  Safety  - Short- 
comings of  Approach 
o Current  Approach 

o Some  Examples  of  Current  Inspection  Problems 

o Summary 

USAF  Aircraft  Accidents  and  Mishaps 

Of  a total  of  3,824  occurences  over  a 15-year  period, 
1,664  were  attributed  to  logistics  involving  airframe, 
engines  and  accessories.  The  remainder  were  attributed 
to  operations,  miscellaneous,  weather  and  unknown 
causes  . 

Distribution  of  Engine  Caused  Aircraft  Accidents  and 
Mishaps 

Engine  failure  causes  had  an  approximate  breakdown  of; 
40%  primary  structure,  40%  pumps,  controls,  etc.,  and 
20%  sub-system  structure. 

Causes  of  Engine  Failures  (Structural) 

A structural  failure  cause  breakdown  for  engines  shows 
approximate  proportions  of;  27%  material  defects,  manu- 
facturing or  assembly  flaws  and  design  deficiencies. 
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27%  fatigue,  12%  stress  rupure,  10%  excessive  creep, 
10%  control  malfunction,  8%  corrosion,  and  6%  foreign 
object  damage. 

Crack  Initiation /Growth  and  Failure  Mechanisms 


(Structural) 

Of  1226  major  cracking/failure  incidents  over  a 21- 
month  period,  approximately  60%  were  fatigue  caused, 
35%  attributed  to  corrosion,  4%  of  overload,  and  1%  to 
unknown  causes.  Another  study  of  64  incidents  attri- 
butes failure  causes  to;  65%  corrosion,  24%  fatigue, 
10%  overload,  and  1%  unknown  causes. 

Cracking  and  Failure  Origins 

A study  of  64  major  c r ac k i ng / f a i 1 u r e problems  on 
military  aircraft  over  a 6-month  period  yielded  the 
following  approximate  proportion  of  causes;  60%  pre- 
existing material  and  fabrication  quality  deficiencies, 
12%  service-induced  corrosion  pits  and  scratches,  16% 
design  deficiencies  such  as  poor  design  details  with 
sharp  corners,  and  12%  unknown  causes. 

Some  Basic  Aspects 

o Aircraft  often  contain  pre-existing  flaws;  nicks, 
scratches,  cracks,  tears,  etc. 
o Flaws  grow  in  size  with  time  (flight  hours), 
o Part  failure  occurs  when  the  flaw  reaches  critical 
size. 

o The  time  to  failure  (life)  depends  on  the  initial 
flaw  size  and  the  crack  growth  rate. 

Factors  which  influence  the  initial  flaw  size 
are:  material  process,  manufacturing  process, 
quality  control  and  inspection. 

Factors  which  influence  crack  growth  rates  are: 
usage  severity,  stress  levels,  environment  and 
material . 

o Aircraft  safety  can  be  achieved  by: 

A design  and  maintenance  plan  which  ensures 
crack  detection  and  repair  before  cracks  reach 
critical  sizes  (the  safe  crack  growth  approach) 
or 


o 
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A design  which  can  tolerate  a single  part  or 
member  failure  without  causing  loss  of  aircraft 
(i.e.  the  fc.il-safe  approach). 

Prior  to  1970,  the  Military  required  neither 
approach . 

o The  Fail-Safe  approach  has  generally  been  used  for 
commercial  aircraft. 

The  Pre-1969/70  Approach 

o Design  and  Analysis 

"Safe-life"  fatigue  design  and  analysis  for 
safety  and  durability  required  that  a predicted 
mean  life  for  a failure  occurrence  distribution 
be  established,  a scatter  factor  for  four  (4) 
is  then  applied  to  conservatively  estimate  the 
design  life,  i.e.,  the  design  life  is  one- 
fourth  (1/4)  of  the  predicted  mean  life. 

The  mean  life  predictions  are  based  on  unflawed 
lab  specimen  data  and  Miner's  rule  of  cumula- 
tive damage. 

The  scatter  factor  is  assumed  to  account  for 
the  effect  of  initial  quality  (flaws)  and  the 
environment  and  variations  in  material  proper- 
ties. 

No  crack  growth  analysis  was  required, 
o Testing 

Fatigue  testing  (for  safety  and  durability) 
involved  block  spectrum  loading. 

The  test  duration  v/as  four  (4)  lifetimes. 
The  full-scale  test  article  was  a produc- 
tion aircraft. 

There  was  no  schedule  requirement. 

The  acceptance  criterion  was,  "no  failure 
in  four  (4)  lifetimes". 

There  was  no  damage  tolerance  testing 
required . 

o Force  Management 

The  assumed  "safe-life"  of  the  force  was  1/4  of 
the  test  life. 

Maintenance  actions  (inspections  and  modifica- 
tions) were  based  on  the  test  time  occurrence 
of  failure  (when  in  the  4 lifetimes)  divided  by 
four  (4)  . 
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The  maintenance  times  were  adjusted  by  fleet 
tracking  records. 

It  was  assumed  that  the  1/4  test  life  limit  and 
scheduled  inspections  would  allow  safe  opera- 
tion. 

Shortcomings  of  the  Pre-1969/70  Approach 

o It  did  not  preclude  the  improper  use  of  high- 

strength,  flaw-sensitive  materials. 

anall  critical  cracks  in  these  materials  can 

jeopardize  safety  and/or  result  in  expensive 
inspection  programs. 

o It  did  not  always  inhibit  the  use  of  excessively 

high  stress  levels. 

o Early  catastrophic  failure  can  negate  determination 
of  actual  economical  operational  life. 

o In-service  inspections  will  not  protect  uninspect- 
able  structure. 

o Safety  of  monolithic  structures  is  necessarily 
assured  by  using  fatigue  test  results  and  a scatter 
factor  of  four  (4 ) . 

o It  did  not  preclude  the  use  of  stress  corrosion 
sensitive  materials. 

o It  lacked  accountability  of  load  ordering  effects. 

o It  was  often  deficient  in  critical  area  identifica- 
tion. 

o It  lacked  visibility  of  crack  growth  character- 

istics necessary  for  an  economical  and  effective 
force  management  program. 

o It  often  resulted  in  late  test  results,  and  thus 

costly  retrofits. 

Current  MIL-STD-1530A  Approach 

o TTie  "Safe-life"  approach  has  been  abandoned  and 

safety  and  durability  requirements  decoupled. 

o Safety-Aircraft  will  be  designed  to  be  damage 

tolerant  by  either; 

Fail-safe  or 

Safe  Crack  Growth  criteria. 

o Durability  - Aircraft  will  be  designed  such  that 

the  economic  life  is  greater  than  the  required 
design  life. 

| 
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o Fail  Safe  Design  - The  structure  is  designed  to 

contain  a single  member  failure  without  loss  of  the 
aircraft.  Limitations  of  this  approach  are: 

Early  detection  of  member  failure  is  required. 
It  is  generally  heavier  than  the  safe  crack 
growth  approach. 

Fail-safety  deteriorates  with  onset  of  adjacent 
member  cracking. 

o Safe  Crack  Growth  Design  - The  structure  is 

designed  (and  inspected)  so  that  the  maximum 
expected  initial  damage  will  not  grow  to  critical 
size  during  service  use.  Limitations  to  this 
approach  are  that  safety  is  dependent  on: 

A fracture  control  program  which  is  necessary 
to  ensure  that  no  initial  flaws  greater  than 
the  size  assumed  in  design  will  escape  detec- 
tion. 

An  individual  aircraft  tracking  program  with 
recorded  service  history  is  required. 

Fracture  critical  areas  must  be  accurately 
identif ied. 

Some  Examples  of  Current  USAF  Inspection  Problems 

These  problems  include  detection  of: 

o Onset  of  general  area  cracking  in  fail-safe  cargo/ 
transport/tanker  aircaft. 

o Single  member  failure  in  nonflight  or  ground  evi- 
dent fail-safe  structures. 

o Inner  tab  (hidden  splice  element)  cracking  in 

shiplap  wing  planks  and  stringer  cracking  in  skin 
and  stringer  wings. 

o Small  crack  detection  at  fastener  holes  without 
fastener  removal. 

o Very  small  crack  detection  at  fastener  holes  with 
fasteners  removed. 

o Very  shallow  surface  and  small  internal  cracks  at 
specific  locations  in  gas  turbine  engine  parts. 

Concluding  Remarks 

o Aircraft  safety  is  vitally  dependent  on  reliable 

in-service  inspections. 

The  inspection  problem  is  generally  most 
difficult  w'  th  slow  crack  growth  designs. 
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- The  inspection  problem  is  not  severe  on  older 
ai rcr af t . 

o Required  detection  capabilities  vary  with  specific 
applications,  however,  they  are  typically  as 
follows  for  airframes: 

Fastener  holes  with  fastener  removed;  0.05"  or 
less  corner  flaws. 

Fastener  holes  with  fastener  in-place;  0.10" 
corner  flaws. 

Inner  layer  and/or  interior  stringers;  0.20"  to 
0.50"  through-the-thickness  flaws. 

At  free  edges;  0.05"  corner  flaws  for  hi- 
performance  engines: 

Surface  flaws;  0.01"  or  less  in  depth. 

Internal  flaws;  0.02"  or  less  in  diameter. 
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BASED  ON  A STUDY  OF  64  MAJ( 
CRACKING/FAILURE  PROBLEMS 
ON  MILITARY  A/C  (6  MONTH 
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